From user-return-18353-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Fri Jul 1 08:49:15 2011 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 668E94A30 for ; Fri, 1 Jul 2011 08:49:15 +0000 (UTC) Received: (qmail 70047 invoked by uid 500); 1 Jul 2011 08:49:12 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 69773 invoked by uid 500); 1 Jul 2011 08:48:51 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 69747 invoked by uid 99); 1 Jul 2011 08:48:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2011 08:48:40 +0000 X-ASF-Spam-Status: No, hits=3.2 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of teddyyyy123@gmail.com designates 209.85.161.172 as permitted sender) Received: from [209.85.161.172] (HELO mail-gx0-f172.google.com) (209.85.161.172) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Jul 2011 08:48:35 +0000 Received: by gxk19 with SMTP id 19so1533320gxk.31 for ; Fri, 01 Jul 2011 01:48:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=sLDD1UzCsTC5cnfr857wdnK0Jvcb+67zgF+gKyCqCVc=; b=NZ1YjNY49YC9kKU80tSlq27br09mqKzrU91k2LG4pMZep/upZaUBzeMjoaaY0ZtIte Y4Uwr5ETy4HEMkhNqvi/raXOiBa0fYVqvikO+xz1VWk1ZaTClBe4FY58dO6oi2SXfL/W 7V5wWI+mABhHIbM2S7lTJJlSxic33jPK2ROo8= MIME-Version: 1.0 Received: by 10.236.79.70 with SMTP id h46mr3612319yhe.240.1309510094487; Fri, 01 Jul 2011 01:48:14 -0700 (PDT) Received: by 10.236.69.130 with HTTP; Fri, 1 Jul 2011 01:48:14 -0700 (PDT) In-Reply-To: References: <9CEB91B6-7E06-477E-A16D-A653E1FA0991@thelastpickle.com> <4F6E7263-84D4-4847-BA3E-91FD465A0D27@thelastpickle.com> Date: Fri, 1 Jul 2011 01:48:14 -0700 Message-ID: Subject: Re: custom reconciling columns? From: Yang To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=20cf300fb07903002404a6fe1430 --20cf300fb07903002404a6fe1430 Content-Type: text/plain; charset=ISO-8859-1 https://issues.apache.org/jira/browse/CASSANDRA-2843 thanks Yang On Fri, Jul 1, 2011 at 12:09 AM, Sylvain Lebresne wrote: > I think it's an interesting solution. And we can probably avoid the two > getTopLevelColumns flavors with at bit a refactor. Let's open a ticket > however, > because this is starting to be off-topic for the user mailing list. > > -- > Sylvain > > On Fri, Jul 1, 2011 at 12:44 AM, Yang wrote: > > ok, I kind of found the magic bullet , but you can only use it to shoot > your > > enemy close really close range :) > > > > for read path, the thrift API already limits the output to a list of > > columns, so it does not make sense to use maps in the internal > operations. > > plus the return CF on the read path is not going to be modified/shared by > > any other threads, so synchronization is not necessary. so > > the solution is to modify ColumnFamilyStore so that getTopLevelColumns > takes > > a returnCF param, instead of always constructing it inside with > > ColumnFamily.create(). > > so only read path behavior is changed. > > in read path, we pass in a FastColumnFamily implementation, which uses an > > ArrayList internally to store sorted columns, and do binary search to > insert > > , and merge to addAll(column). > > I tried out this, it's about 50% faster on rows with 3000 cols. > > > > Jonathan: do you think this is a viable approach? the only disadvantage > is a > > slight change to getTopLevelColumns so we have 2 flavors of this method > > Thanks > > Yang > > > > On Wed, Jun 29, 2011 at 5:51 PM, Jonathan Ellis > wrote: > >> > >> On Tue, Jun 28, 2011 at 10:06 PM, Yang wrote: > >> > I'm trying to see whether there are some easy magic bullets for a > >> > drop-in > >> > replacement for concurrentSkipListMap... > >> > >> I'm highly interested if you find one. :) > >> > >> -- > >> Jonathan Ellis > >> Project Chair, Apache Cassandra > >> co-founder of DataStax, the source for professional Cassandra support > >> http://www.datastax.com > > > > > --20cf300fb07903002404a6fe1430 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable https://issu= es.apache.org/jira/browse/CASSANDRA-2843

thanks
Yang

On Fri, Jul 1, 2011 at 12:09 AM, Syl= vain Lebresne <sylvain@datastax.com> wrote:
I think it's an interesting solution. And we can probably avoid the two=
getTopLevelColumns flavors with at bit a refactor. Let's open a ticket = however,
because this is starting to be off-topic for the user mailing list.

--
Sylvain

On Fri, Jul 1, 2011 at 12:44 AM, Yang <teddyyyy123@gmail.com> wrote:
> ok, I kind of found the magic bullet , but you can only use it to shoo= t your
> enemy close really close range :)
>
> for read path, the thrift API already limits the output to a list of > columns, so it does not make sense to use maps in the internal operati= ons.
> plus the return CF on the read path is not going to be modified/shared= by
> any other threads, so synchronization is not necessary. so
> the solution is to modify ColumnFamilyStore so that getTopLevelColumns= takes
> a returnCF param, instead of always constructing it inside with
> ColumnFamily.create().
> so only read path behavior is changed.
> in read path, we pass in a FastColumnFamily implementation, which uses= an
> ArrayList internally to store sorted columns, and do binary search to = insert
> , and merge to addAll(column).
> I tried out this, it's about 50% faster on rows with 3000 cols. >
> Jonathan: do you think this is a viable approach? the only disadvantag= e is a
> slight change to getTopLevelColumns so we have 2 flavors of this metho= d
> Thanks
> Yang
>
> On Wed, Jun 29, 2011 at 5:51 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>>
>> On Tue, Jun 28, 2011 at 10:06 PM, Yang <teddyyyy123@gmail.com> wrote:
>> > I'm trying to see whether there are some easy magic bulle= ts for a
>> > drop-in
>> > replacement for concurrentSkipListMap...
>>
>> I'm highly interested if you find one. :)
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra supp= ort
>> http://www.d= atastax.com
>
>

--20cf300fb07903002404a6fe1430--