Let's develop an open-source media archive standard

From: Fred Cisin <cisin_at_xenosoft.com>
Date: Wed Aug 11 18:02:20 2004

On Wed, 11 Aug 2004, Steve Thatcher wrote:
> to re-iterate, the separate sections I am proposing let the emulator do
> this very easily. Embed the format and the data together as pyhsical
> information and then the matter of extracting a file for emulation
> becomes difficult and not universal.

That's an important point.

Each person has different uses for it. Steve and I want
something that is practical for file extraction (I could
replace the sector read code in XenoCopy with some file
parsing,...)

And for regeneration, there is such a thing as TOO MUCH
data. If you try to replicate a disk, where every single
bit/flux transition is specified, it is not feasable.
There is a reason for "GAP" fields! If you try to
replicate the disk, you will not be able to recreate
some of the mangled bytes in the write splice fields.
Unless you are trying to replicate sopy-protection,
you would be WAY better off storing JUST the data,
and the specs of the format, and recreating the format
instead of trying to replicate it!
Instead, a 360K disk should have 360K of data, whether
as 360K of binary bytes, or as 720K+ of ASCII hex dump,
PLUS just the information that it is 9 512 byte sectors
per track. Any exceptions to "standard" formatting
could be documented in the header, OR within the data
as necessary.
For example, a DS Kaypro disk should have the head number
field of the sectors on the second side wrongly set to 0.

But OTHER people, such as the CAPS project feel a need to
replicate every flux transition.


Therefore, since this is an extensible spec,
I propose that it have a multi-layer structure:
(I realize that this will grievously offend those
who are insisting on a single invariable structure!)

Mundane mode:
Header with physical format data, including a flag
of whether the data is in binary or ASCII hex dump,
followed by a stream of all the user data.

In mundane mode, with trivial processing, a machine
with an FDC could format the disk and write the
data to it, for disks with no abnormalities.
File size in binary mode would be only a few blocks
(header) larger than the original disk.
File size in ASCII hex dump mode would be 3 times
the size of the original disk.
Either file size could be reduced a bit through RLE.

In mundane mode, one CP/M format could be trivially
converted to another in cases where the number of
reserved tracks and records per block match, just
by stepping on the bytes per sector and sectors per
track data!

In mundane mode, reading alien disks consists of
parsing the directory data on the disk enough to
determine where each file starts and ends.


Minimal quirks mode:
Similar to mundane mode, but header could specify
(IF KNOWN) any logical interleave, and simple
variants from normal formatting (such as the
Kaypro DS)

Medium quirks mode:
Specify the formatting for each and every sector

Maximum quirks mode:
Specify every miswritten byte, the contents of every
gap and sector header field, etc. With minor processing
and sent to a CatWeasel/Option board, etc. would produce
a decent duplicate of the disk.

A binary form of maximum quirks mode could look like the raw
bit stream from a Catweasel/Option board.
But for practical study, ASCII hex dump mode would be preferable.
(look at the TE.COM program of the Option board)
Received on Wed Aug 11 2004 - 18:02:20 BST