OT Vast amounts of NASA data in weird formats (Long)

From: McFadden, Mike <mmcfadden_at_cmh.edu>
Date: Wed Jun 14 15:19:15 2000

Now that the subject line somewhat matches the topic.
NASA and other unnamed government agencies collect "LOTS" of data. They
have lots of tapes, some ASCII card images on tape, and other raw formats.
Now when a project is winding down do you think anyone "cares"/spends money
to transfer any data into newer formats. The original software that was
written probably handles the data and works OK. I understand that lots of
LANDSAT data has cloud cover obscuring it. Do you save even the
"apparently" worthless stuff? The first thing any engineer/programmer tries
is to pack the data onto the tape as efficiently as possible. 12 bit
pixels get stored 2 pixels in every 3 bytes. Everybody used different
methods to handle uneven numbers of pixels, including padding, and
truncation. How do you decide how the data was stored. I'm sure the paper
that documents the data format is detached from the physical tape.
The short answer is that when data is collected no one has any idea of what
eventually may be done with the data. They only expend as much time,
energy, and money as the initial project seems to justify. I will try and
find my copy of a GAO report that I purchased on the magnitude of the data
storage/retention problem. They had pictures of doors in the data vault
being held open by stacks of tapes.

Mike
mmcfadden_at_cmh.edu
Data Hound extraordinary, I'm burying my floppy disks for posterity. Maybe
we should use gold floppy disks will not oxidize and absorb moisture in any
short term interval.
Received on Wed Jun 14 2000 - 15:19:15 BST