db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kurt Huwig <k.hu...@iku-ag.de>
Subject Re: Question about scalability
Date Wed, 28 May 2008 20:55:49 GMT
Hi,

I am using Derby for exactly this purpose: logfiles of a mailserver. 
This triggers the following bug quite frequently:

https://issues.apache.org/jira/browse/DERBY-2991

The only working solution right now is to synchronize() all INSERT and 
SELECT operations to the log table. This certainly means that you 
cannot log mails while you create reports, so you need some queueing.

Am Mittwoch 28 Mai 2008 schrieb sin(EaTing),:
> Thanks Bryan!
>
> However I guess I have missed to say I have a requirement which needs
> frequently search out these records to form something like a visit
> history report. These reports is a little bit complex, which may need
> to group data by different criteria like the country of these users
> and usage among different months or years.
>
> I think obviously just recording all visits in a single table should
> be the easiest way to write out the select statement. However the
> performance may be low, even if indexes are properly created, and the
> data set will accumulate to be very large if I use a single table.
>
> So others showed me some way to just discard these old data or
> achieve them to file systems to maintain a small footprint of the
> active data set. But the achieved data will be another obstacle to
> select out data easily.
>
>
> On Wed, May 28, 2008 at 9:50 PM, Bryan Pendleton
> <bpendleton@amberpoint.com>
>
> wrote:
> > I am working on a database design which can record down each of the
> > visit
> >
> >> to my web application.
> >
> > I think that's a great use for Derby, and you should be able to be
> > very successful, if you are careful with your design. The more
> > information about the details that you can share with the
> > community, of course, the more useful the feedback will be.
> >
> > One technique that I think works well with this sort of
> > time-oriented data is to structure it as an ongoing series of
> > tables, of similar schema, with each table holding a segment of
> > time.
> >
> > For example, you might determine that you can hold 1 week's worth
> > of data in a table, so (since there are 52 weeks in a year), you
> > name the tables for 2008 something like:
> >
> >  CREATE TABLE WEBLOG_2008_1 (...)
> >  CREATE TABLE WEBLOG_2008_2 (...)
> >  CREATE TABLE WEBLOG_2008_3 (...)
> >  CREATE TABLE WEBLOG_2008_4 (...)
> >
> > and you store the first week's data into WEBLOG_2008_1, and the
> > second week's data into WEBLOG_2008_2, and so forth.
> >
> > At any given instant there is exactly 1 "current" table for this
> > week's data, to which you are appending rows, and some collection
> > of "historical" data which you can query as your needs arise.
> >
> > thanks,
> >
> > bryan



-- 
Mit freundlichen Grüßen

Kurt Huwig (Vorstand)                     Telefon 0681 / 3 72 00 36 - 50
http://www.iku-ag.de/                     Telefax 0681 / 3 72 00 36 - 59

iKu Systemhaus AG, Altenkesseler Straße 17/Gebäude C1, 66115 Saarbrücken
Amtsgericht: Saarbrücken, HRB 13240
Vorstand: Kurt Huwig            Aufsichtsratsvorsitzender: Jan Bankstahl

GnuPG 1024D/99DD9468 64B1 0C5B 82BC E16E 8940  EB6D 4C32 F908 99DD 9468

Mime
View raw message