flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From EdwardKing <zhan...@neusoft.com>
Subject Why flume can't start?
Date Wed, 14 May 2014 01:25:51 GMT
My OS is CentOS 5.8, I use hadoop2.2 and I download apache-flume-1.4.0-bin.tar.gz,I use following
command to install flume

$ mv apache-flume-1.4.0-bin.tar.gz /opt
$ tar -xzf /opt/apache-flume-1.4.0-bin.tar.gz
$ ln -s /opt/apache-flume-1.4.0 /opt/flume
Define the FLUME_HOME environment variable:
Export FLUME_HOME=/opt/flume
Export PATH=${FLUME_HOME}/bin:${PATH}

Create the directory that will act as the Flume conf directory
$ mkdir /home/hadoop/flume/conf

Copy the needed files into the conf directory:
$ cp /opt/flume/conf/log4j.properties /home/hadoop/flume/conf
$ cp /opt/flume/conf/flume-env.sh.template /home/hadoop/flume/conf/flume-env.sh

Then edit flume-env.sh and set JAVA_HOME.

Then I create the following file as agent1.conf under /home/hadoop/flume
agent1.sources = netsource
agent1.sinks = logsink
agent1.channels = memorychannel
agent1.sources.netsource.type = netcat
agent1.sources.netsource.bind = localhost
agent1.sources.netsource.port = 3000
agent1.sinks.logsink.type = logger
agent1.channels.memorychannel.type = memory
agent1.channels.memorychannel.capacity = 1000
agent1.channels.memorychannel.transactionCapacity = 100
agent1.sources.netsource.channels = memorychannel
agent1.sinks.logsink.channel = memorychannel

Start a Flume agent:
[hadoop@master flume]$ flume-ng agent --conf conf --conf-file 10a.conf --name agent1
Info: Sourcing environment configuration script /home/hadoop/flume/conf/flume-env.sh
Info: Including Hadoop libraries found via (/home/software/hadoop-2.2.0/bin/hadoop) for HDFS
access
Info: Excluding /home/software/hadoop-2.2.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar from
classpath
Info: Excluding /home/software/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar
from classpath
+ exec /home/software/jdk1.7.0_02/bin/java -Xmx20m -cp '/home/hadoop/flume/conf:/opt/flume/lib/*:/home/software/hadoop-2.2.0/etc/hadoop:/home/software/hadoop-2.2.0/share/hadoop/common/lib/activation-1.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/asm-3.2.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/avro-1.7.6.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/avro-mapred-1.7.6.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-cli-1.2.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-codec-1.4.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-digester-1.8.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-el-1.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-io-2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-lang-2.5.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-math-2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/commons-net-3.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/guava-11.0.2.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/hadoop-annotations-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/hadoop-auth-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jersey-core-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jersey-json-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jersey-server-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jettison-1.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jetty-6.1.26.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jsch-0.1.42.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jsp-api-2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/junit-4.8.2.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/log4j-1.2.17.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/paranamer-2.3.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/paranamer-2.6.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/servlet-api-2.5.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/xmlenc-0.52.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/xz-1.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/lib/zookeeper-3.4.5.jar:/home/software/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0-tests.jar:/home/software/hadoop-2.2.0/share/hadoop/common/hadoop-nfs-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/common/jdiff:/home/software/hadoop-2.2.0/share/hadoop/common/lib:/home/software/hadoop-2.2.0/share/hadoop/common/sources:/home/software/hadoop-2.2.0/share/hadoop/common/templates:/home/software/hadoop-2.2.0/share/hadoop/hdfs:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/asm-3.2.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0-tests.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/hdfs/jdiff:/home/software/hadoop-2.2.0/share/hadoop/hdfs/lib:/home/software/hadoop-2.2.0/share/hadoop/hdfs/sources:/home/software/hadoop-2.2.0/share/hadoop/hdfs/templates:/home/software/hadoop-2.2.0/share/hadoop/hdfs/webapps:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/asm-3.2.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/guice-3.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/hadoop-annotations-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/hamcrest-core-1.1.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/javax.inject-1.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/junit-4.10.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib/xz-1.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-api-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-client-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-common-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-common-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-site-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib:/home/software/hadoop-2.2.0/share/hadoop/yarn/lib-examples:/home/software/hadoop-2.2.0/share/hadoop/yarn/sources:/home/software/hadoop-2.2.0/share/hadoop/yarn/test:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/junit-4.10.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/file::/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/lib-examples:/home/software/hadoop-2.2.0/share/hadoop/mapreduce/sources:/home/software/hadoop-2.2.0/contrib/capacity-scheduler/*.jar'
-Djava.library.path=:/home/software/hadoop-2.2.0/lib/native org.apache.flume.node.Application
--conf-file 10a.conf --name agent1

[hadoop@master flume]$ 

Why flume can't start? Where is wrong with my above steps? How to make flume running? Thanks.



---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)

is intended only for the use of the intended recipient and may be confidential and/or privileged
of 
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication
is 
not the intended recipient, unauthorized use, forwarding, printing,  storing, disclosure or
copying 
is strictly prohibited, and may be unlawful.If you have received this communication in error,please

immediately notify the sender by return e-mail, and delete the original message and all copies
from 
your system. Thank you. 
---------------------------------------------------------------------------------------------------
Mime
View raw message