commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elifarley Callado Coelho Cruz <elifar...@gmail.com>
Subject Re: Very big arrays in Java and Commons Collections
Date Thu, 11 Aug 2005 11:04:51 GMT
Or you could use a BDB database:

http://today.java.net/pub/a/today/2004/08/24/sleepy.html

http://www.sleepycat.com/products/je.shtml

Excerpt:

"
Of course, when I go through this list, and try to explain Berkeley DB
to people, they usually respond with comments like:
 So, it's, like, what? A well-behaved HashMap? 
To which I usually reply, "well, yeah, sort of. It's actually a little
more like a well-behaved TreeMap, but you get the idea."
At its core, a Berkeley DB database is really just a b-tree with a
persistent file format and nice transactional semantics thrown in. It
makes sense to use it, or something like it, in situations when all of
the following are true:
 * You need to store objects in an indexed collection.
 * You only need a small number of fairly simple ways to retrieve the objects.
 * You don't need ad-hoc queries.
 * You need transactional insert/remove/update operations on your storage.
 * You need persistence to disk.
 * You need a very small footprint, and for the database to be in-process.
 * You don't need to have multiple processes or applications access
the database.
"

Cheers,
Elifarley

On 8/9/05, Wade Chandler <hwadechandler-apache@yahoo.com> wrote:
> This sounds like a really good project proposal
> actually for a Collections based file mapping library
> or a large ArrayList...obviously unable to have a
> toArray method except for sections of the file.
> Anyways, I don't know of any direct wrappers for this,
> but you can use java.nio.FileChannel obtaining it from
> a java.io.RandomAccess file and then using the map
> method to map regions of the file into memory.  This
> would allow you to read and write bytes, floats,
> doubles, ints, arrays of all etc to this mapped region
> making a larger area obviously than allowed memory
> without using much or hardly any depending on the
> needed efficiency concept.  You could make use of this
> to store objects as well depending on how you are
> storing them.  You can store them as arrays of bytes
> and read and write them out to the mapped area giving
> you more space to work....all conceptual of course.
> Might help you though.
> 
> Wade
> 
> --- Sergey Khenkin <skhenkin@gmail.com> wrote:
> 
> > Hi All,
> >
> > Does anybody know of a mean to work with very large
> > arrays that don't
> > fit in memory and are cahced to disk when necessary?
> > Is there a proper type of collection in Commons
> > Collection?
> > Thanks.
> >
> > --
> > Best regards,
> > Sergey Khenkin
> > mailto:skhenkin@gmail.com
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > commons-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > commons-user-help@jakarta.apache.org
> >
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
> 
>

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message