Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 94480 invoked from network); 1 Feb 2010 16:33:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 1 Feb 2010 16:33:21 -0000 Received: (qmail 14706 invoked by uid 500); 1 Feb 2010 16:33:20 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 14651 invoked by uid 500); 1 Feb 2010 16:33:20 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 14641 invoked by uid 99); 1 Feb 2010 16:33:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 16:33:20 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of elf2001@gmail.com designates 209.85.218.218 as permitted sender) Received: from [209.85.218.218] (HELO mail-bw0-f218.google.com) (209.85.218.218) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2010 16:33:11 +0000 Received: by bwz10 with SMTP id 10so386193bwz.35 for ; Mon, 01 Feb 2010 08:32:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=pnGjJpfXpN6/qGXA+DzNmXJ8XklibHKZSyk8OMHlioU=; b=HpInu48/VGPe73GPIgazQUYatjGo5Dg58fQLskTL6OOQ1ALkrcwcdPT5GM5Kq9gyOX 2JWWB82JJDH8DrBnIspk4iNJLAhr638h8RzGVNQwcmpfz5byPMOyjTX3YRCEmT2r+vpi 9UhtoUsh16u3gse2CqnU6ZJMbAr23ZlS+3TmE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=xJr9cJ/PFlo4TQ7MNxKFX57yAnV6haWDv3eDMLvMXdeZxRaMsx2oMVqqEaprxSwcvq 8i2z6EOpundnDqrN3lTY1oUT0bNQMTTHOmDiddT0zK+FAsTHi0F+B9C586SXrqXmLxsD ihbgQdBxijn2ZuT+EIg8LdNSV7tpxJ50r2e44= MIME-Version: 1.0 Received: by 10.204.141.78 with SMTP id l14mr3270555bku.50.1265041969761; Mon, 01 Feb 2010 08:32:49 -0800 (PST) In-Reply-To: <4B6700EB.7030906@woop.es> References: <4B6700EB.7030906@woop.es> Date: Mon, 1 Feb 2010 18:32:49 +0200 Message-ID: <4a29064f1002010832h4b20aa66xe0ddeb5bc7133104@mail.gmail.com> Subject: Re: Best way to store 2^32 IPs in CouchDB From: Elf To: user@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Did you plan to handle IPv6 in future versions of your program? :) 2010/2/1 Santi Saez : > > Hi, > > I'm doing some initial tests with CouchDB, trying to store 2^32 IP addresses > (approximately 4.3 billions of documents). > > Documents have only required fields: _id and _rev, but I've noticed that the > minimum space occupied by each document is approximately 3.7KB, so I need > +14TB disk space only for the basic scheme without any extra field (using IP > as unique identifier in integer format). > > Note that playing with a simple Python script and a binary data file, this > data can be stored in 16GB space (each IP 4 = bytes * 2 ^32 addresses). > > Is it possible to optimize the disk space for what I'm trying to do using > CouchDB? Perhaps disabling "something", compressing, or changing _rev field > format/size.. thanks!! > > I haver read the manual for CouchDB perfomance, but I didn't get it: > > http://wiki.apache.org/couchdb/Performance > > Regards, > > -- > Santi Saez > http://woop.es > -- ---------------- Best regards Elf mailto:elf2001@gmail.com