couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: Best way to store 2^32 IPs in CouchDB
Date Mon, 01 Feb 2010 16:56:16 GMT
Well, 2^32 of anything is 4GiB per byte stored. So, minimum of four
bytes and you're at 16GiB. Even with just 1KiB overhead you're at
4TiB.

I'm left wondering why you would want to store a list of numbers in
the first place.

HTH,
Paul Davis

On Mon, Feb 1, 2010 at 11:27 AM, Santi Saez <santisaez@woop.es> wrote:
>
> Hi,
>
> I'm doing some initial tests with CouchDB, trying to store 2^32 IP addresses
> (approximately 4.3 billions of documents).
>
> Documents have only required fields: _id and _rev, but I've noticed that the
> minimum space occupied by each document is approximately 3.7KB, so I need
> +14TB disk space only for the basic scheme without any extra field (using IP
> as unique identifier in integer format).
>
> Note that playing with a simple Python script and a binary data file, this
> data can be stored in 16GB space (each IP 4 = bytes * 2 ^32 addresses).
>
> Is it possible to optimize the disk space for what I'm trying to do using
> CouchDB? Perhaps disabling "something", compressing, or changing _rev field
> format/size.. thanks!!
>
> I haver read the manual for CouchDB perfomance, but I didn't get it:
>
> http://wiki.apache.org/couchdb/Performance
>
> Regards,
>
> --
> Santi Saez
> http://woop.es
>

Mime
View raw message