Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: local policy)
From: "Hiller, Dean" <Dean.Hiller@nrel.gov>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Wed, 29 May 2013 10:33:20 -0600
Subject: Re: Is there anyone who implemented time range partitions with
 column families?
Thread-Topic: Is there anyone who implemented time range partitions with
 column families?
Thread-Index: Ac5cijsQb9lJLSgHSvKeU/D526RdjQ==
Message-ID: <CDCB88EA.298F4%Dean.Hiller@nrel.gov>
In-Reply-To: <CDCB837C.298E7%Dean.Hiller@nrel.gov>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.3.4.130416
acceptlanguage: en-US
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Something we just ran into with compaction and timeseries data.  We have
60,000 virtual tables(playorm virtual tables) inside ONE CF.  This
unfortunately hurt our compaction with LCS since it can't be parallized
for a single tier.  We should have had 10 CF's called data0, data1, data2
=8A.data9 such that we could be running 10 compactions in parallel.

QUESTION: I am assuming 10 compactions should be enough to put enough load
on the disk/cpu/ram etc. etc. or do you think I should go with 100CF's.
98% of our data is all in this one CF.

Thanks,
Dean

On 5/29/13 10:06 AM, "Hiller, Dean" <Dean.Hiller@nrel.gov> wrote:

>Nope, partitioning is done per CF in PlayOrm.
>
>Dean
>
>From: cem <cayiroglu@gmail.com<mailto:cayiroglu@gmail.com>>
>Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>"
><user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
>Date: Wednesday, May 29, 2013 10:01 AM
>To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>"
><user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
>Subject: Re: Is there anyone who implemented time range partitions with
>column families?
>
>Thank you very much for the fast answer.
>
>Does playORM use different column families for each partition in
>Cassandra?
>
>Cem
>
>
>On Wed, May 29, 2013 at 5:30 PM, Jeremy Powell
><jeremym.powell@gmail.com<mailto:jeremym.powell@gmail.com>> wrote:
>Cem, yes, you can do this with C*, though you have to handle the logic
>yourself (other libraries might do this for you, seen the dev of playORM
>discuss some things which might be similar).  We use Astyanax and
>programmatically create CFs based on a time period of our choosing that
>makes sense for our system, programmatically drop CFs if/when they are
>outside a certain time period (rather than using C*'s TTL), and write
>data to the different CFs as needed.
>
>~Jeremy
>
>On Wed, May 29, 2013 at 8:36 AM, cem
><cayiroglu@gmail.com<mailto:cayiroglu@gmail.com>> wrote:
>Hi All,
>
>I used time range partitions 5 years ago with MySQL to clean up data much
>faster.
>
>I had a big FACT table with time range partitions and it was very is to
>drop old partitions (with archiving) and do some saving on disk.
>
>Has anyone implemented such a thing in Cassandra? It would be great if we
>have that in Cassandra.
>
>Best Regards,
>Cem.
>
>