Tape dumping programs for Unix/Linux...

From: Douglas H. Quebbeman <dquebbeman_at_acm.org>
Date: Thu May 2 18:06:35 2002

Stan sayeth:

> > No. And I see no useful purpose in including that in the format.
> > If the read was recovered, you have the data; surely you would
> > *not* want to recreate a tape and make a previously marginal
> > block, marginal again on a clone?
>
>
> NOOOO! It's *IMPORTANT* to know if there was a question about the
> accuracy of the data! That's *WHY* some tape drives even *tell*
> you there was a recovered error ... so the user could use that
> information!

I do that during the first dozen-or-so reads of the tape...

> It's like getting a recovered parity error on memory.
> Should you know about it? DAMN STRAIGHT YOU SHOULD!

heh... for years, any IBM compatible PC would, on detection
of a parity error, deliberately crash. Doesn't matter why,
I just lots my stuff...

> Without recording that information, you also can't answer
> important questions like: is the orginal tape in good shape?

Sure, *make note of it*, but I don't necessarily view the
image file as the only correct place to put that info. But
it's not a bad idea, just a bit Brobdignagian...

If I was Tim Shoppa and was doing analog reads of tapes so
far gone that you've got only one try to get it right, sure.

But like I said before, traditional methods shouldn't be used
on these tapes.

> > > - ability to flag if a particular tape record had a hard error
> > > while reading it
> >
> > No (I can say this even though I'm about to ask): do you mean an
> > unrecoverable error, or one that was corrected by the hardware
> > (as opposed to being corrected by software)? Either way, if the
> > error was corrected, see my response above.
>
> If it was corrected, then there was no error (other than a
> recovered read, already correct recorded via the earlier bit :)
>
> I meant a real, live, actual error. (E.g., missing oxide)
>
> When recording the contents of a tape it's **IMPORTANT** to know
> that there was an error there. If you simply skip the erorr and
> keep reading, you've lost data.
> Did you lose original user data? Possibly, and you've definitely
> lost data about the tape (i.e., the fact there was an error).
>
> By recording the fact an error occurred, along with any data
> your tape drive/OS *may* have sent you, you preserve the ability
> to act wisely in the future. For example, by recording the
> fact there was an error at megabyte 100 in a 102 MB file,
> the user would have confidence that when that file is eventually
> extracted, the first 99 MB of it is in fine shape.

Y'all must encounter a lot more marginal tapes than I do. The
ones I can't read have gone all the way down to where they will
require an analog read. If I can read it digital at all, I'll
reread the dang tape until I get ever record error-free. The
wet read is most useful in this application.

> > > - ability to flag if an End-of-tape marker/indicator was seen
> > > while reading the the current record (again, not all
> > > hardware/OS's support getting that information)
> >
> > No, and again, I can't see how this data would be used to recreate
> > the tape or would be needed by an emulator. It's kind of like asking
> > if a particular hardware emulator also emulates memory parity errors
> > so that you can see memory parity errors in the logs. Why??
>
> With 9-track tapes, software writing to the tapes generally watches
> for an indication that the end-of-tape marker (a short silver
> reflective strip) has been seen. At that time, it can prepare
> to do a reel-switch. Although some people feel they can backspace
> and write reel-switch info prior to the EOT marker on a 9-track
> tape, that's technically not safe (and opens up the possibility
> of getting what's called (in 1979) "crap in the gap"). Anyway,
> the ANSI (?) standard for 9 track tapes specifies a minimum length
> of tape be available for writing after the EOT marker ... a length
> that should be sufficient (even with modest multiple outstanding
> write-ahead requests) to write the reel switch information.

I think you're missing my point- I don't believe there are any
9-track drives that split physical records across reels of tape.

If I'm wrong, please correct me.

> On at least one OS, Burroughs MCP, one could detect that EOT at
> read time. Then, if one is doing multiple read-ahead requests,
> one can moderate them appropriately.

Yes, yes, yes, of course the EOT is detected while reading. But
it happens *between* records, not *inside* them.

> Ok...the short answer: it's information. You don't throw away
> information without a darn good reason. :)

Hey, if physical records *can* be split bwteen reels, then you
are absolutely right.

> > > - overall header at start of file, recording information about the tape
> >
> > No, although I did consider suggesting some extensions like this.
> > One problem I have encountered with some old CDC tape formats
>
> Sorry...if you want a professional program, not something like
> that toy cptape, you *WILL* have this information.
>
> Been there, worked on that, for over 30 years. Trust me...you
> *want* all the information you can get.

Actually, all I want is that the system software and applications
can use the data from the tape without detecting its coming from
some place other than a physical tape. So far that goal has not
required the extreme mesaures discussed here.

> > Anyway, I encode that kins of descriptive information (what kind
> > of computer, encoding, #tracks, etc) in the filename, or in the
> > name of the directory containing the file, or in a README.
>
> If two items of data are separated, what happens to them?
> Yep. Should be 'nuff said :)

Well, Raymond and I agree that tarballs work well for this, as
well as other such methods of encapsulation.

> > > I have a program called "tapedisk", which reads an arbitrary tape
> >
> > This sounds familiar... is it on the web somewhere?
>
> The MPE V version was/is (can't recall which), and was free.
> The MPE/iX version is a product from our company.

I was thinking of some Linux thing... I've only a few hours
of HP3000 experience. Looked like a nice extension of the
2000 Access system, at first...

> > So you really do want to recreate bad tapes as bad tapes...
> >
> > Why?
>
> No...I want to know that a problem existed, and where that
> problem was.
>
> Again...this is basic, vital, and indisputable. You *NEED* to
> know if there was a problem with the tape, and if you're trying to
> extract a particular file you need to know where any problems
> are in that file. Otherwise, you'll be lying to the user.
>
> If you know where within a file an error was found, then you've
> got a chance at recovering most of the rest of the data successfully.
> If you know an error occurred *somewhere* in a file, but not
> where, then you have a large pile of bits which *might* be trash,
> and *might* be useful ... but neither you or the user knows.
> And, if you don't know where the errors were (and how many there
> were), you can't reliably restore the data. That data might
> be banking information that your company is being audited on;
> it might be data that could clear someone of a crime...you simply
> don't know.
>
> Data is important...that's why we call it data and not junk :)

We're not disagreeing about this, we just have a different
time-and-place attitude about it.

> > This is some what I store as a "header" at the start of the
> > > overall output:
> >
> > ...
> >
> > Yeah, this info is README fodder... why on earth program that?
>
> Uh, 'cuz someone had to do it and build a reliable, thorough
> program?

I'm not arguing that it's of no value, but I'm looking at
cost of effort expended vs. return on investment. I used
to always believe in building for the worst-case, but these
days it's rarely feasible. The people writing the checks
don't want to pay for features they don't think they'll use.

> > > As you can see...a lot of info, but all necessary to faithfully
> > > reproduce the tape later and/or to understand *why* the file might
> > > (or can't) be used to completely reproduce the tape.
> >
> > Uh, again, can't understand why you'd need to recreate bad tapes.
>
> Again...you're *NOT RECREATING BAD TAPES*.

I understand. Recreating tapes from images is not a criterium.

> > You might try hooking up with the emulator community and
> > ask your questions there.
>
> This being ClassicCmp, I thought they might be here, too :)

One or two of us, at least!
 
> Besides, this isn't an emulator question so much as a data
> preservation question.

Related issues, to be sure, but not necessarily identical ones.

I don't think we're really disagreeing here; you circle
the block to the left, I circle to the right. We both
make it to where we want to go.

Regards,
-dq
Received on Thu May 02 2002 - 18:06:35 BST