From user-return-12809-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Mon Sep 20 13:33:34 2010 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 98490 invoked from network); 20 Sep 2010 13:33:33 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 20 Sep 2010 13:33:33 -0000 Received: (qmail 77849 invoked by uid 500); 20 Sep 2010 13:33:31 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 77820 invoked by uid 500); 20 Sep 2010 13:33:27 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 77807 invoked by uid 99); 20 Sep 2010 13:33:26 -0000 Received: from Unknown (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Sep 2010 13:33:26 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of wout.mertens@gmail.com designates 209.85.215.180 as permitted sender) Received: from [209.85.215.180] (HELO mail-ey0-f180.google.com) (209.85.215.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Sep 2010 13:33:04 +0000 Received: by eya25 with SMTP id 25so2212734eya.11 for ; Mon, 20 Sep 2010 06:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:content-type:mime-version :subject:from:in-reply-to:date:content-transfer-encoding:message-id :references:to:x-mailer; bh=Uav59wQ101AQrgOGjTT+X/yk1AFojmlQU8fq0vS3xEc=; b=a4y37v4kAtnWL3oOQ0r2KSZqs5aniF+OTu2TMDmY7vCQ1KM0/k/gp5FxaO5dx/ZH0f vpEgxcbXuXJSAhXSGnGtJeFb/HSPFKR1r1A3Z0vCc6rMz/KU+9DCeHurDRgk1U8t8lK7 sAWWDgIabIHgRq5qzeTiOfWYTWUnnqwAIfcCk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; b=QfFjKO8qf3CwJhm908XjZgz17tJd/S1Dznr8j3ghT8x02m5pcuSGsajkt3kT8Rudt/ OlDhOx39tK3GIDxj2dgWPt7SQiBSkwEHj7OTuXIAN8RaglesJtPptmiV+qWAc0butGCl 41AFmCOtx/YzNsPVbsfHflCq5TIPTOCtWfwok= Received: by 10.213.10.84 with SMTP id o20mr2774406ebo.50.1284989564531; Mon, 20 Sep 2010 06:32:44 -0700 (PDT) Received: from [192.168.2.13] (94-224-251-197.access.telenet.be [94.224.251.197]) by mx.google.com with ESMTPS id z55sm10978150eeh.3.2010.09.20.06.32.19 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 20 Sep 2010 06:32:21 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Apple Message framework v1081) Subject: Re: Which filesystem is best for deploying couchdb and why? From: Wout Mertens In-Reply-To: Date: Mon, 20 Sep 2010 15:32:18 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <503AA132-6B13-4C89-8F81-AC0B6F0D0AB7@gmail.com> References: <79513BE4-3FE0-4395-9D7C-859FDB1DB09C@gmail.com> To: user@couchdb.apache.org X-Mailer: Apple Mail (2.1081) X-Virus-Checked: Checked by ClamAV on apache.org On Sep 19, 2010, at 9:42 , Metin Akat wrote: > Hmm, then I start thinking that btrfs is really better than ext4, > especially if I want to do things like copy the files in order to > deploy (fast) another instance with already build view indexes. Well, ext4 is considered more stable than btrfs, that has to count for = something in your reasoning :-) How would btrfs help with fast copies? You would still need to transfer = the files to another system, no? > And would the couchdb files compress well? Yup... Wout. >=20 > On Sat, Sep 18, 2010 at 12:23 PM, Wout Mertens = wrote: >> Everything except the 4KB blocksize I would say. While ZFS and Btrfs = are very different on-disk, they are both Copy-On-Write filesystems with = extents, compression and cheap snapshots. I don't know how the 4KB = blocksize settings translates onto Btrfs. >>=20 >> They are both really suited for the append-only workload CouchDB = presents. >>=20 >> Wout. >>=20 >> On Sep 18, 2010, at 9:49 , Metin Akat wrote: >>=20 >>> What part of this blog post is relevant to btrfs? >>>=20 >>> On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson = wrote: >>>> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies = wrote: >>>>> Wow, thanks for the thought out writeup! >>>>>=20 >>>>=20 >>>> here's a blog post = http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/ >>>>=20 >>>>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds = wrote: >>>>>=20 >>>>>> Disclaimer: I'm no file systems expert. >>>>>>=20 >>>>>> I recommend something with extents otherwise you might take a big >>>>>> performance hit while couch deletes old db files after = compaction. >>>>>> Compression sounds cool as long as you can do it really fast (are >>>>>> there setups where this happens in hardware?). >>>>>>=20 >>>>>> reiserfs: >>>>>> According to wikipedia it "still uses the big kernel lock (BKL) =97= a >>>>>> global kernel-wide lock" which makes performance on multiple = cores >>>>>> suffer. >>>>>> It's big benefit, as I always understood it, is being able to = pack >>>>>> smile files together into single blocks. You will likely not have = lots >>>>>> of small files with Couch :-P >>>>>>=20 >>>>>> xfs: >>>>>> Delayed allocation might be a big performance win with a Couch. = Since >>>>>> outstanding writes are committed together in chunks and then = fsync'd >>>>>> all together I bet this feature would do good things for Couch >>>>>> performance. >>>>>>=20 >>>>>> ext(3|4) >>>>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well = as >>>>>> the multiblock allocator should make it much better than ext3. = You >>>>>> also get extents. >>>>>>=20 >>>>>> btrfs/zfs: >>>>>> Some of the features of each sound interesting, but nothing that >>>>>> stands out to me as "great for CouchDB". Snapshots and backups = are >>>>>> cool, but Couch is doing this for you already in a sense due to = the >>>>>> way the btree is appended: CouchDB documents are, in a sense, >>>>>> copy-on-write. Checksumming is cool if you think it's important = for >>>>>> your data integrity. If you want snapshots for backup you can = always >>>>>> use CouchDB replication. >>>>>>=20 >>>>>> If you run any tests I'd be very, very interested in seeing your = results. >>>>>>=20 >>>>>> -Randall >>>>>>=20 >>>>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat = wrote: >>>>>>> I'm sure almost everybody out there is using ext4/3 (including = me), >>>>>>> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some = of >>>>>>> them have very appealing feature-sets (like compression for = example, >>>>>>> and we all know how greedy is couchdb for disk space). >>>>>>> And I know that for example btrfs is not yet "recommended for >>>>>>> production". But its time is coming. =46rom what I see, Ubuntu = 10.10 >>>>>>> works flawlessly on btrfs. >>>>>>> So I'd be happy if we have some discussion on the topic, instead = of >>>>>>> "everybody uses ext4, just use it" kind of stuff :). >>>>>>> Couchdb was "alpha software" for years, and we all used it in >>>>>>> production, so we are not afraid of alpha/beta software, as long = as >>>>>>> it's good :) >>>>>>=20 >>>>>=20 >>>>>=20 >>>>>=20 >>>>> -- >>>>> http://www.readwriteweb.com/about#tyler >>>>>=20 >>>>> Ask me anything ! >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> -- >>>> Chris Anderson >>>> http://jchrisa.net >>>> http://couch.io >>>>=20 >>=20 >>=20