On Wed, Jul 02, 2003 at 08:20:30AM -0400, Rock wrote:
> I am beginning research into disk archiving of all of our paper based
> sales and customer service data. My intention is not to go back and
> re-enter all the old data but rather start at some point archiving all
> the new data. I want to stay with a Linux solution. Our hardware
> supplier, CDW, says they have archiving solutions and will be happy to
> discuss them with me, but I am sure that I will pay through the nose for
> their "solution".
>
> Does anyone have any experience with data archiving using Linux that you
> will be willing to share?
>
> Everything related to a customer or potential customer is referenced by
> a customer number so I think that would simplify the solution. I want
> everything including but not limited to the following: Initial
> correspondence, proposals, art work, sales contracts, A/R documents,
> payments and all correspondences.
You'll need to index it by date as well, and include the type of data
and its format. Metadata.
One thing is not clear: either this data is getting entered into
computer in the first place, or you're talking about paper documents.
For paper documents, they'll have to be imaged first, before archiving.
Either way, I think you can just store this stuff (with some metadata,
like custno, transdate and doctype) in MySQL or PostgreSQL. You
mentioned having users dump this stuff into some subdirectory scheme.
Good luck. I have my doubts you can make people do this consistently.
However, you just might be able to make them dump the stuff into a
single directory. You could have a cron job sweep the stuff out and into
the various subdirectories, if you have metadata available. Getting
users to supply the metadata may be a chore, though. After data has been
swept out, another cron job could add the data to a database (as above).
Compression could be a problem. You can store it compressed, but no
matter what you do, you're going to have to have a front end that allows
users to find and display the data they want. The metadata will show
what kind of data is contained in this particular "lump", and your front
end will use the appropriate application to display it.
And you'll need a single person responsible to fix problems with the
system. Things like, the date for the lump was wrong. Or the use
specified the wrong type of data, so it can't be displayed properly,
etc.
Maybe you could work out a web-based interface, where the user logs on
and says, "I want to store this data." It forces him/her to enter the
metadata. Then it includes an "Upload" button that sucks the data off
his/her machine and puts it in the proper place.
Do I know of a solution that does all this? Oh heck no. I'm just
thinking out loud. ;-}
Paul
This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 18:32:19 EDT