incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Pérez <bi...@tuenti.com>
Subject Suggestion for changing the way for setting the cluster
Date Tue, 09 Mar 2010 11:22:37 GMT
Right now we are setting the destination cluster by setting the tags
field of the record, and then in RecordUtil by applying a regular
expression, that may be slow specially if we process tons of records.

I would suggest to include the destination cluster in the key object.
I think makes more sense, it's an easy change, plus we don't fill
chukwa records with unneeded data. I attach a tentative patch that
will do this. It's much easier to control now the destination cluster
from the mapper, plus the record object is much cleaner now. And the
regular expression is done only once per chunk, not once per record,
potentially increasing speed.

What do you think? Should I open a ticket or this has no sense?

-- 
Guille -ℬḭṩḩø- <bisho@tuenti.com>
:wq

Mime
View raw message