chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: setting collector on a remote machine
Date Wed, 08 Sep 2010 15:34:44 GMT
Yes, you can run the collector on any host with network access to
HDFS. Verify that HADOOP_HOME and HADOOP_CONF_DIR are properly set in
chukwa-env.sh. Also verify that you can connect to hadoop from the
command line with those environment variables set.

On Tuesday, September 7, 2010, Shreya Chakravarty
<shreya_chakravarty@persistent.co.in> wrote:
> Hi,
>
> We want to setup a collector on a non HDFS machine
> Is that possible? Will the collector be able to send data to hdfs, we tried setting the
writer.hdfs.filesystem property to point to the hdfs master, but getting the following error:
>
>
> 2010-09-07 05:53:11,153 INFO main ChukwaConfiguration - chukwaConf is /home/hadoop/chukwa-0.4.0/conf/
> 2010-09-07 05:53:12,721 INFO main root - initing servletCollector
> 2010-09-07 05:53:12,727 INFO main PipelineStageWriter - using pipelined writers, pipe
length is 2
> 2010-09-07 05:53:12,751 INFO Thread-9 SocketTeeWriter - listen thread started
> 2010-09-07 05:53:12,755 INFO main SeqFileWriter - rotateInterval is 300000
> 2010-09-07 05:53:12,755 INFO main SeqFileWriter - outputDir is /chukwa/logs/
> 2010-09-07 05:53:12,755 INFO main SeqFileWriter - fsname is iaas-1-0-16-3e-b7-d1-29:54310
> 2010-09-07 05:53:12,756 INFO main SeqFileWriter - filesystem type from core-default.xml
is org.apache.hadoop.hdfs.DistributedFileSystem
> 2010-09-07 05:53:12,860 ERROR main SeqFileWriter - can't connect to HDFS, trying default
file system instead (likely to be local)
> java.io.IOException: No FileSystem for scheme: iaas-1-0-16-3e-b7-d1-29
>         at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1375)
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>         at org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter.init(SeqFileWriter.java:123)
>         at org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter.init(PipelineStageWriter.java:88)
>         at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.init(ServletCollector.java:112)
>         at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440)
>         at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
>         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>         at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:685)
>         at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
>         at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
>         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>         at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
>         at org.mortbay.jetty.Server.doStart(Server.java:224)
>         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>         at org.apache.hadoop.chukwa.datacollection.collector.CollectorStub.main(CollectorStub.java:121)
>
> Regards
> Shreya
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is the property
of Persistent Systems Ltd. It is intended only for the use of the individual or entity to
which it is addressed. If you are not the intended recipient, you are not authorized to read,
retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.
>

Mime
View raw message