Re: [SLUG] A question of databases

From: Paul M Foster (paulf@quillandmouse.com)
Date: Mon Dec 06 2004 - 22:30:48 EST


On Mon, Dec 06, 2004 at 11:09:44AM -0500, Chuck Hast wrote:

> After seeing some more comments on databases I have some questions
> which I hope after getting answers will allow me to determine what route
> I want to go.
>
> First I know nil about DB stuff. I am trying to learn though.
>
> Here is the scenario:
> I have multiple applications gathering data on wireless networks, there
> may be up to 10 machines in a vehicle (possibly even more depending
> on how many networks we are trying to "can you hear me now?")
> capturing data, each machine is doing a sample at no faster than 1
> capture/sec, at the same time it is also doing round trip time testing
> as fast as the packets can make the round trip (ping like). Each one
> of these functions is dumping data with up to 25 fields/record. On
> a normal coverage test drive day we will capture up to over 100,000
> records/application. There will be 2 db tables/provider (one for RF
> environment data the other for the round trip time data and the RF
> enviro data at that point in time) except for CDMA providers who have
> two separate captures being recorded, the same as above but there
> is a additional machine which initiates a data call every 45 seconds
> and times the amount of time needed to obtain a working channel
> then hanges up or gets a error code and starts over again.
>
> At the same time there will be at least 1 machine looking at the db
> and mapping the data as it comes in, that is so the person monitoring
> the data can stop the drive test and go back and check a questionable
> area if needed. There may be other mappers looking at the db as the
> data is being captured.
>
> So based on the above info (probably not enough but hope it sheds
> light on what direction I need to go) what db should I look at using for
> this application. The data needs to get in there fast enough so the
> mappers can update the view without the vehicle having gone too far
> down the road. I want this to be a Linux machine obviously.
>
> At the end of the data the db will be copied to another machine for
> other people to map all of the data the next day while the drive testing
> continues.

Either MySQL or PostgreSQL should do, though I prefer PostgreSQL. Since
these are client-server architectures, db updates from your sources will
be queued, so speed of data influx shouldn't (hopefully) be a problem.
The back end analysis should also benefit from a client-server
architecture, since the process querying the database can do so without
disturbing the inbound data stream. You may have to tweak the
implementation, depending on your exact circumstances.

You can write your software in something like C and use the MySQL or
PostgreSQL libraries to do the actual heavy lifting.

Paul

-----------------------------------------------------------------------
This list is provided as an unmoderated internet service by Networked
Knowledge Systems (NKS). Views and opinions expressed in messages
posted are those of the author and do not necessarily reflect the
official policy or position of NKS or any of its employees.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 20:33:06 EDT