commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lian Ort" <lian...@googlemail.com>
Subject Re: flexible byte-stream storage? [vfs] [cache] ??
Date Wed, 14 Mar 2007 09:22:05 GMT
Hi Paul,
my experience is for the native bdb. We use it on aix and windows.
I have no experience with bdb-xml.

The experience with native bdb is very good. we use it in a data migration
project for pre-building the target database with bdb.

The db2 database we create this way has several gigabytes. the size of
individual entities created
ranges from some under 100 bytes to big entities with blobs in it of several
thousand kilobytes.

Greetings,
Lian

On 3/14/07, Paul Libbrecht <paul@activemath.org> wrote:
>
> Lian,
>
> well, I did try BDB-xml long ago for another purpose and it was
> receiving xml-documents as... strings! very non-scalable. Do you have a
> better experience ?
> BDB is native, that's a bit annoying but could be a good candidate.
> An API pointer would be nice.
>
> paul
>
>
> Lian Ort wrote:
> > Hello Paul,
> > did you check Berkeley DB (BDB;
> > http://www.oracle.com/database/berkeley-db/index.html) ?
> > Its kinda usable like a HashMap. You are looking for a solution
> > to store Objects as Bytestreams and to get them back via a key. This
> > sounds
> > like a HashMap
> > and this is also one of the way Berkeley DB is usable. BDB only handles
> > bytestreams, no concrete
> > java objects. You can store any sized bytestream in it and the library
> is
> > fast.
> > Greetings,
> > Lian
> >
> >
> > On 3/14/07, Thorbjørn Ravn Andersen <thunderaxiom@gmail.com> wrote:
> >>
> >> Paul Libbrecht skrev  den 21-02-2007 21:22:
> >> > Thanks for the hint of a directory split in a way similar to a
> >> > hash-table, I hadn't thought about it and it would solve the horrible
> >> > 50'000 files/directory.
> >> > Choosing the host platform is not an option though... so the
> >> > filesystem also not.
> >>
> >> > Overall these 50'000 only bother if you look at it.
> >> >
> >> It may also be a performance problem, depending on whether the
> directory
> >> is searched linearly or sequentially for a named file.
> >> > Where I am rather making hopes is for the small-sized chunks for
> which
> >> > making a full-file is a waste on several file systems which have
> >> > minimal sizes of 512 bytes, and some 4Kb...
> >> > Of course I could write my library for this, with java.nio most
> >> > probably, but I was hoping VFS or some other library to help me
> there.
> >> >
> >> Could you elaborate on your scenario.  Perhaps the usage patterns lend
> >> towards a given pattern.
> >>
> >> --
> >>   Thorbjørn
> >>
> >>
> >
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message