couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wout Mertens <wout.mert...@gmail.com>
Subject Re: Which filesystem is best for deploying couchdb and why?
Date Mon, 20 Sep 2010 13:32:18 GMT
On Sep 19, 2010, at 9:42 , Metin Akat wrote:

> Hmm, then I start thinking that btrfs is really better than ext4,
> especially if I want to do things like copy the files in order to
> deploy (fast) another instance with already build view indexes.

Well, ext4 is considered more stable than btrfs, that has to count for something in your reasoning
:-)

How would btrfs help with fast copies? You would still need to transfer the files to another
system, no?

> And would the couchdb files compress well?

Yup...

Wout.

> 
> On Sat, Sep 18, 2010 at 12:23 PM, Wout Mertens <wout.mertens@gmail.com> wrote:
>> Everything except the 4KB blocksize I would say. While ZFS and Btrfs are very different
on-disk, they are both Copy-On-Write filesystems with extents, compression and cheap snapshots.
I don't know how the 4KB blocksize settings translates onto Btrfs.
>> 
>> They are both really suited for the append-only workload CouchDB presents.
>> 
>> Wout.
>> 
>> On Sep 18, 2010, at 9:49 , Metin Akat wrote:
>> 
>>> What part of this blog post is relevant to btrfs?
>>> 
>>> On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson <jchris@apache.org> wrote:
>>>> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <tyler@pdxbrain.com>
wrote:
>>>>> Wow, thanks for the thought out writeup!
>>>>> 
>>>> 
>>>> here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/
>>>> 
>>>>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <randall.leeds@gmail.com>wrote:
>>>>> 
>>>>>> Disclaimer: I'm no file systems expert.
>>>>>> 
>>>>>> I recommend something with extents otherwise you might take a big
>>>>>> performance hit while couch deletes old db files after compaction.
>>>>>> Compression sounds cool as long as you can do it really fast (are
>>>>>> there setups where this happens in hardware?).
>>>>>> 
>>>>>> reiserfs:
>>>>>> According to wikipedia it "still uses the big kernel lock (BKL) —
a
>>>>>> global kernel-wide lock" which makes performance on multiple cores
>>>>>> suffer.
>>>>>> It's big benefit, as I always understood it, is being able to pack
>>>>>> smile files together into single blocks. You will likely not have
lots
>>>>>> of small files with Couch :-P
>>>>>> 
>>>>>> xfs:
>>>>>> Delayed allocation might be a big performance win with a Couch. Since
>>>>>> outstanding writes are committed together in chunks and then fsync'd
>>>>>> all together I bet this feature would do good things for Couch
>>>>>> performance.
>>>>>> 
>>>>>> ext(3|4)
>>>>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well
as
>>>>>> the multiblock allocator should make it much better than ext3. You
>>>>>> also get extents.
>>>>>> 
>>>>>> btrfs/zfs:
>>>>>> Some of the features of each sound interesting, but nothing that
>>>>>> stands out to me as "great for CouchDB". Snapshots and backups are
>>>>>> cool, but Couch is doing this for you already in a sense due to the
>>>>>> way the btree is appended: CouchDB documents are, in a sense,
>>>>>> copy-on-write. Checksumming is cool if you think it's important for
>>>>>> your data integrity. If you want snapshots for backup you can always
>>>>>> use CouchDB replication.
>>>>>> 
>>>>>> If you run any tests I'd be very, very interested in seeing your
results.
>>>>>> 
>>>>>> -Randall
>>>>>> 
>>>>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <akat.metin@gmail.com>
wrote:
>>>>>>> I'm sure almost everybody out there is using ext4/3 (including
me),
>>>>>>> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some
of
>>>>>>> them have very appealing feature-sets (like compression for example,
>>>>>>> and we all know how greedy is couchdb for disk space).
>>>>>>> And I know that for example btrfs is not yet "recommended for
>>>>>>> production". But its time is coming. From what I see, Ubuntu
10.10
>>>>>>> works flawlessly on btrfs.
>>>>>>> So I'd be happy if we have some discussion on the topic, instead
of
>>>>>>> "everybody uses ext4, just use it" kind of stuff :).
>>>>>>> Couchdb was "alpha software" for years, and we all used it in
>>>>>>> production, so we are not afraid of alpha/beta software, as long
as
>>>>>>> it's good :)
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> http://www.readwriteweb.com/about#tyler
>>>>> 
>>>>> Ask me anything <http://tumble.pdxbrain.com/ask>!
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Chris Anderson
>>>> http://jchrisa.net
>>>> http://couch.io
>>>> 
>> 
>> 


Mime
View raw message