incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <>
Subject Re: speeding up demux?
Date Mon, 10 May 2010 23:40:30 GMT
On Mon, May 10, 2010 at 4:39 PM, Ariel Rabkin <> wrote:
> Can you say a bit about where your bottleneck is?  Is there one reduce
> that's taking a very long time? Can you check the logs and see which
> datatype that reducer is dealing with?  There was some discussion of
> this on JIRA recently; consensus is that our current partitioner works
> well if you have a wide variety of datatypes, none of which is too
> big, and badly if you have one or two datatypes with lots of data in
> each.

I forgot to add -- the JIRA you should follow is

We'd love to get feedback on what a more sensible approach would be
for handling your use case.


Ari Rabkin
UC Berkeley Computer Science Department

View raw message