cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Drew Dahlke <drew.dah...@bronto.com>
Subject Re: Map Reduce support
Date Mon, 28 Jun 2010 12:17:37 GMT
The difference is noticeable but small. I did a test just reading data
in from Cassandra on our cluster & dumping it to a csv file. Pure map
reduce was going at ~17k records/sec versus ~15k from Pig. There is
overhead to using Pig, but it'll reduce your development time & make
for more readable code if it suits your needs.

On Sun, Jun 27, 2010 at 9:53 AM, Atul Gosain <atul.gosain@gmail.com> wrote:
> Thanks for the information Drew and Jonathan.
> Is there any difference in performance while using Pig compared to MapReduce
> directly on data store ?
> I will do the experiments with both of them though in some time.
>
> On Fri, Jun 25, 2010 at 5:46 PM, Drew Dahlke <drew.dahlke@bronto.com> wrote:
>>
>> The cassandra column family input format will go over a an entire
>> column family sending a slice of a row into a mapper at a time. From
>> there there's a lot you can do. As far as how you aggregate data
>> together, I'd suggest experimenting with the latest version of Pig
>> which thankfully supports the new input format. It gives you a
>> SQL'esque syntax for manipulating the data and is probably the easiest
>> way to experiment.
>>
>> On Thu, Jun 24, 2010 at 11:01 AM, Atul Gosain <atul.gosain@gmail.com>
>> wrote:
>> > Hi
>> >   What kind of Map Reduce support is provided for Cassandra ?
>> > Can i get some columns from different rows and then aggregate them up
>> > together. Its basically aggregation of statistics for various devices
>> > connected to a network manager. Is it a right kind of use case to be
>> > supported by MR ?
>> > Thanks
>> > Atul
>
>

Mime
View raw message