Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of rajkumar.w93@gmail.com
 designates 209.85.161.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=Lpf8fFmrlgHeqwqIUIpT6nS6B2Bi2zMowXpCbV3QeQky7J1JvSivlbhVAanagL4Oyy
         m8BatYseXcvNzqF7LICIbBocfNQyXOciSR+Vin8ARgkKOTDQNBQ4EdYsmZFkj/dXHWii
         RVkD84i6fVw0EUuqnCHYIXq67ndvPcPGAtI5g=
MIME-Version: 1.0
Sender: rajkumar.w93@gmail.com
In-Reply-To: <AANLkTikBazEBkPi6UQzjXkx_vX+=0Y8PkpgymkkzWdJ_@mail.gmail.com>
References: <AANLkTimMp3nSuPdAW-OhWoVXbmNd-Dai+0DC25GH1NVM@mail.gmail.com>
	<AANLkTi=2j_x5RauO47W7HBzBQ1zu=-WxmKE92PC-fKL-@mail.gmail.com>
	<AANLkTikBazEBkPi6UQzjXkx_vX+=0Y8PkpgymkkzWdJ_@mail.gmail.com>
Date: Sat, 5 Feb 2011 12:28:54 +0530
Message-ID: <AANLkTimS4O86hpdBe5TEcEWEEtj+0TuwkJqYx8=7pZ5_@mail.gmail.com>
Subject: Re: Merging the rows of two column families(with similar attributes)
 into one ??
From: Ertio Lew <ertiop93@gmail.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Yes, a disadvantage of more no. of CF in terms of memory utilization
which I see is: -

if some CF is written less often as compared to other CFs, then the
memtable would consume space in the memory until it is flushed, this
memory space could have been much better used by a CF that's heavily
written and read. And if you try to make the thresholds for flush
smaller then more compactions would be needed.


On Sat, Feb 5, 2011 at 11:58 AM, Ertio Lew <ertiop93@gmail.com> wrote:
> Thanks Tyler !
>
> I could not fully understand the reason why more no of column families
> would mean more memory.. if you have under control parameters like
> memtable_throughput & memtable_operations which are set per column
> family basis then you can directly control & adjust by splitting the
> memory space between two CFs in proportion to what you would do in
> single CF.
> Hence there should be no extra memory consumption for multiple CFs
> that have been split from single one??
>
> Regarding the compactions, I think even if they are more the size of
> the SST files to be compacted is smaller as the data has been split
> into two.
> Then more compactions but smaller too!!
>
>
> Then, provided the same amount of data, how can greater no of column
> families could be a bad option(if you split the values of parameters
> for memory consumption proportionately) ??
>
> --
> Regards,
> Ertio
>
>
>
>
>
> On Sat, Feb 5, 2011 at 10:43 AM, Tyler Hobbs <tyler@datastax.com> wrote:
>>
>>> I read somewhere that more no of column families is not a good idea as
>>> it consumes more memory and more compactions to occur
>>
>> This is primarily true, but not in every case.
>>
>>> But the caching requirements may be different as they cater to two
>>> different features.
>>
>> This is a great reason to *not* merge them.=A0 Besides the key and row c=
aches,
>> don't forget about the OS buffer cache.
>>
>>> Is it recommended to merge these two column families into one ?? Though=
ts
>>> ?
>>
>> No, this sounds like an anti-pattern to me.=A0 The overhead from having =
two
>> separate CFs is not that high.
>>
>> --
>> Tyler Hobbs
>> Software Engineer, DataStax
>> Maintainer of the pycassa Cassandra Python client library
>>
>>
>