incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Pérez <>
Subject Extend demux aliases to reducers?
Date Wed, 10 Mar 2010 18:02:14 GMT
Right now the conf/chukwa-demux-conf.xml file defines a set of aliases
for mappers, so you just specify a Name of the record, and the
appropriate mapper class is used. I would like to do more or less the
same for reducers.

My mapper generates different chukwa records with different keys. For
example record "network" and record "disk". But the reducer class is
the same, just makes some calculations. Right now if I specify the
same reducer class in the mapper, the output file is the name of the
reducer, so both types of records end in the same file.

I would like to just make key.setReduceType(ReduceAlias). That
ReduceAlias will be the name of the output file, but the actual class
for the reduce will be defined in the config, the same way we do for
mappers. Or maybe -again- setting the output file name in the key as I
suggested for the cluster destination, but I think the first will be
more general, but may tie code to config (a mapper sets reduce type
"foo" in code, and needs "foo" to be defined in the config) . What do
you think? What will be the best approach?

Guille -ℬḭṩḩø- <>

View raw message