lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Storing GData Feeds -- need your help!
Date Thu, 23 Nov 2006 07:42:46 GMT
Simon,
Try BerkeleyDB.  It's free, it's fast, it's got a super-simple API, it let's you tune caching
easily, etc.

Otis

----- Original Message ----
From: Simon Willnauer <simon.willnauer@googlemail.com>
To: java-dev@lucene.apache.org
Sent: Wednesday, November 22, 2006 5:38:26 AM
Subject: Storing GData Feeds -- need your help!

Hello java-devs,

I assume you guys are all quiet experienced java developers and
familiar with xml. After SummerOfCode I do have a bit more relaxed
time to implement one of the most important parts of gdata server -->
the storage layer. During summer of code I used DB4O which is GPL so I
implemented bridge code to save a bit of time.
I actually don't wanna make this implementation as a requirement for
lucene gdata because it is gpl and not ASF licensed. I evaluated some
kind of persistence technologies like xml databases, RDBMS including
some techniques to store xml in a generic and efficient way but I
don't have any really satisfying results.

The very reduced requirements for a storage are It should be
transactional, scalable and fast on reading the content. I experienced
the main problem on the server side is to parse the content to
recreate the object representation as this costs a lot of time and
performance. One the one hand there is no need for an object
representation while a entry of feed is requested by a simple http GET
so I could skip the parsing on a read request and fire the result into
the output stream straight away without parsing it at all. But is that
the way it should be?! One the one hand this could offer a quiet good
performance on the other hand this could be a problem for new features
in the future. Implementing the feed / entry as a proxy and if some
processing is needed the content will be parsed is also desirable but
would cost some performance.

During development I used an lucene index as a storage which could be
a considerable alternative as a feed storage without parsing. The
current implementation of the Feed / Entry is an Interface so the
could be storage dependent implementation for that as well.

I'm actually looking for alternatives and suggestions about this
topic, I know you guys have your own schedule but I would really
appreciate some help with that.

if anyone of you have any idea about the license stuff
(http://www.gossamer-threads.com/lists/lucene/java-dev/42253), who I
have to contact and if it is ok for the PMC a short answer would be
great.


best regards Simon

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message