chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <>
Subject Re: Demux depends on conf/chukwa-collector-conf.xml
Date Fri, 05 Mar 2010 21:05:39 GMT
Demux process should not be using writer.hdfs.filesystem.  The original
implementation made a mistake to use ³writer.hdfs.filesystem² as reference
to hdfs.  Instead, demux should pick up the proper hdfs url from
HADOOP_CONF_DIR.  I don¹t have any objection on the suggested workaround,
but less configuration that we need to maintain, the better.

I file a bug for this issue:


On 3/5/10 12:34 PM, "Bill Graham" <> wrote:

> Hi,
> We've packaged our own chukwa rpms to be able to deploy the collector, the
> agent and the data processor each separately. In doing so I realized that the
> data processor (i.e., Demux) has a dependency on
> conf/chukwa-collector-conf.xml as packaged in the distro, since that's the
> only place where where 'writer.hdfs.filesystem' is set.
> The workaround is straight forward enough (just add writer.hdfs.filesystem to
> chukwa-demux.xml), but I thought I'd point out the cross-dependency. Although
> it feels redundant, it seems the proper thing to do would be to include
> writer.hdfs.filesystem in both configs, since both processes have that dep.
> Thoughts?
> thanks,
> Bill

View raw message