I noticed that someone forwarded one of my early messages about the Catweasel
software I was working on to the list. I've made a lot more progress since
then, so in case anyone is interested, below are a few more messages giving
the latest news. Basically, it's working well, even on 8" disks.
My program (cw2dmk) reads disks into a file format that represents decoded
FM or MFM data, with the location of ID address marks noted, but no other
indication of missing clocks or other odd stuff. This might be useful
beyond the two TRS-80 emulators that currently support it, but it isn't
the universal format that folks on this list were trying to design a month
or two ago.
Designing a format like that is an interesting problem. After my experience
reading disks with the Catweasel, I'm not sure that simply recording the
raw sample data is a good archival format. After my program decodes the
FM or MFM data and checks for things like CRC errors and missing/unexpected
data address marks, there are a fair number of errors that go away if
I do a few retries at resampling the track. (Most disks have none, but
they happen often enough to be noticeable when I'm reading a lot of disks.)
So an archiver that just blindly sampled each track once without trying
to decode it might end up recording data that could not be decoded later.
Also, writing a good decoder is a bit of a pain, and some (what appears
to me) black magic to reconstruct the data after bit-shifting has occurred.
It would be nice to do the decoding once, up front, and archive decoded
data. On the other hand, really strangely copy-protected disks from systems
with disk controllers that are partly implemented in software might not
be decodable using a general-purpose algorithm. Maybe it would be best
to *try* to decode the data, do some retries if it can't be decoded, and
archive both the result of decoding and the raw samples just for maximum
safety....
--Tim
========Message 1========
From: mann_at_pa.dec.com (Tim Mann)
Date: Thu, 22 Jun 2000 19:47:28 -0700 (PDT)
To: trs80_at_cs.ubc.ca
Subject: Announcing cw2dmk: read any TRS-80 floppy with a Catweasel
-----
Background: The Catweasel ISA floppy disk controller is an add-in card
for the PC ISA bus. It uses specialized hardware, not a standard floppy
disk controller chip, and it can be programmed to read and write just
about any disk format. Catweasel cards were in short supply for a while,
but as of July 2000 they should be available for purchase again. See
http://www.jschoenfeld.com/ and
http://apd2.tripod.com/catweasl.htm.
I've written a free program for the Catweasel called cw2dmk that can read
any disk written by a TRS-80, including single-density and copy-protected
disks. The disk is saved as a DMK image file, suitable for use in either
the xtrs TRS-80 emulator for Unix, or David Keil's TRS-80 emulator for
MS-DOS. cw2dmk runs on Linux, MS-DOS, or Windows 95/98. You can download
it from
http://www.tim-mann.org/trs80resources.html. Source code is
included. My program uses some code from Michael Krause's Linux drivers
for the Catweasel (which work with Amiga and MS-DOS diskettes).
Besides TRS-80 disks, cw2dmk should be able to read any disk written by
a WD177x/179x or by a standard PC (NEC765-compatible) disk controller.
Currently it's not too useful to have non-TRS-80 disks in DMK format,
but perhaps additional tools that work with this format will become available
in the future. For example, I'd like to write a program that uses a
Catweasel to write a DMK image back to a real floppy disk. This would
provide a nice way of archiving disks from many old machines and making
physical copies when needed.
Another likely future direction for cw2dmk is to be able to write JV1
and JV3 disk images, when the disk being read can be represented in one
of those formats. Those are the formats that Jeff Vavasour's and Matthew
Reed's TRS-80 emulators need.
By the way, I don't get a cut of Catweasel sales. :-)
========Message 2========
From: mann_at_pa.dec.com (Tim Mann)
Date: Thu, 22 Jun 2000 21:33:59 -0700 (PDT)
Newsgroups: comp.sys.tandy, comp.os.cpm
Subject: Re: Announcing cw2dmk: read any TRS-80 floppy with a Catweasel
-----
Oops. In that announcement, I forgot to say "TRS-80 Model I, III, 4,
or 4P". These are the models that are currently supported by the emulators
that use DMK format. Disks written by other computers that used the TRS-80
brand name will probably be readable too, but the images aren't currently
too useful since there is not yet any software that uses DMK format other
than these emulators.
I do plan to read some Model II/12 disks for archival purposes when I
get my 8" drive plugged in, and I'm also going to take a shot at reading
some 3.5" Model 100 disks that someone is mailing me, but I don't have
any results to report on that yet.
========Message 3========
From: Gary Shanafelt <...>
Date: Thu, 22 Jun 2000 22:02:21 -0500
To: trs80_at_cs.ubc.ca
Subject: Re: Announcing cw2dmk: read any TRS-80 floppy with a Catweasel
-----
Tim,
In your last message about the Catweasel board, you mentioned David Keil's
emulator using DMK image files while Jeff Vavasour's and Matt Reed's used JV1
and
JV3 format files. What's the difference? All the emulators call their files
.DSK files; I recently downloaded David's program, set it up with some of the
.DSK files from the Vavasour/Reed emulators, and it worked with them without
further ado. I thought all these .DSK files were a single standard TRS-80
emulator format, but I guess not?
========Message 4========
From: mann_at_pa.dec.com (Tim Mann)
Date: Thu, 22 Jun 2000 21:27:01 -0700 (PDT)
To: Gary Shanafelt <...>
Cc: trs80_at_cs.ubc.ca
Subject: Re: Announcing cw2dmk: read any TRS-80 floppy with a Catweasel
-----
The emulators all give the .DSK extension to their files, but they are
using several different formats, which I've given the names DMK, JV1,
and JV3. Full details are on
http://www.tim-mann.org/trs80/dskspec.html,
but here is a summary.
JV1 can only handle single density disks with directory on track 17. Jeff
Vavasour's Model I emulator works only with JV1 images. Jeff's Model
III/4 emulator doesn't support JV1 images at all, but all the other major
emulators will work with a JV1 image if you give one to them.
JV3 can handle normal single and double density disks, but most forms
of copy-protected disk are not representable. Almost all of the images
on Ira Goldklang's site (www.trs-80.com) are in JV3. All the major emulators
except Jeff's Model I emulator will work with JV3 images. Jeff originated
this format in his Model III/4 emulator.
DMK can handle anything that a TRS-80 disk controller can write to the
disk, including all the copy-protected formats. Even protected game disks
where the program does a Read Track and looks for specific data in the
gaps between sectors can be recorded in DMK format and will run in emulators
that support it. Currently only David Keil's emulator and xtrs support
this format. David originated the format. Ira now has a small collection
of images in DMK format that can't be represented in JV3 format.
========Message 5========
From: mann_at_pa.dec.com (Tim Mann)
Date: Wed, 28 Jun 2000 16:17:10 -0700 (PDT)
To: ...
Subject: Early 8" results with Catweasel on LS-DOS disks
-----
Thought you might be interested to hear that I tried reading a few 8"
disks with a Catweasel last night, using my custom program (cw2dmk).
They were mostly TRS-80 Model II LS-DOS disks with an FM track 0 and MFM
on the rest of the disk.
The FM was easy to read; the MFM required tweaking the thresholds in my
program because the bit-shifting effect (that write precompensation is
meant to adjust for) varies so much from the outer to inner tracks, and
makes a sudden jump backward at track 44 where write precomp was turned
on when the disks were written. I found a setting that worked, but it's
touchy; a few retries were needed on tracks near 43 and 76. I might try
to devise an algorithm that sets the thresholds automatically for each
track by doing a histogram and looking for the exact location of the peaks
and valleys, or I might try doing something that emulates a phase locked
loop in software instead of using fixed thresholds....
The main disks I tried reading were copies of Model II/12 LS-DOS that Lamar
Owen and Fred Dolan sent. The disk Fred sent had CRC errors on about half
the tracks (usually one error per track), no matter what I did. I'm going
to poke a bit more to see if it's a problem with my software or if the
disk is really bad. The disks Lamar sent read OK once I got the thresholds
right.
========Message 6========
From: mann_at_pa.dec.com (Tim Mann)
Date: Mon, 3 Jul 2000 20:30:21 -0700 (PDT)
To: ...
Subject: Re: Early 8" results with Catweasel on LS-DOS disks
-----
An update for those who may be interested:
I found a heuristic to make up for insufficient write-precompensation
(or for flux transitions that were close together moving farther apart
over time). With this added to cw2dmk, I was able to read most of the
8" disks that I have without getting errors, including the one from Fred
Dolan that previously showed CRC errors on about half the tracks.
Here's the idea. When decoding MFM, the interval between each two successive
flux transitions should be either short (2 cycles), medium (3), or long
(4). Because of a physical effect, short transitions that come next to
medium or long ones tend to spread out into their space. So after I classify
a transition, I compute how long it ideally should have been and save
the difference between the real and ideal lengths. I then add k times
the difference to the next transition before classifying it, where k is
a tuning parameter. With k=0, we have the old algorithm. A value of
k around 0.5 to 0.8 made the difficult disks readable. Usually the exact
value wasn't too critical, but on the really bad disks I had to experiment
a bit to find the best setting.
I imagine this technique (or something better) is well known in the disk
drive industry, but it was a kick to me when I thought of it and it worked
so well.
I might go back to some 5.25" disks I have that showed CRC errors when
read on a normal PC controller, and see if this trick will let me read
some of them.
Tim Mann tim.mann_at_compaq.com
http://www.tim-mann.org
Compaq Computer Corporation, Systems Research Center, Palo Alto, CA
Received on Tue Jul 04 2000 - 20:47:17 BST