flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wesley dias <wesleyd...@outlook.com>
Subject RE: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink
Date Tue, 29 Oct 2013 04:53:38 GMT
Thanks Roshan. But issue is now resolved. I compiled flume 1.4 from source using protobuf 2.5
and this solved the below mentioned issue. My guess is that the distribution available for
flume 1.4 from the release site must have been compiled using protobuf 2.4
Date: Mon, 28 Oct 2013 13:41:52 -0700
Subject: Re: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing
to HDFS sink
From: roshan@hortonworks.com
To: wesleydias@outlook.com

HDP already includes hadoop 2.2.0 + Flume 1.4. You'd be better of just using instead of building
and testing that combo out on your own.

On Mon, Oct 28, 2013 at 1:19 PM, wesley dias <wesleydias@outlook.com> wrote:

Hi David,
Thanks for the quick reply.Does this mean that I would need to compile the flume from source
using protobuf 2.5 instead of directly using the flume distribution available for download
from the site? Is flume 1.4 based of protobuf 2.4?

I am not sure of the same hence I am asking these questions.
Date: Mon, 28 Oct 2013 16:11:16 -0400
Subject: Re: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing
to HDFS sink

From: dsinclair@chariotsolutions.com
To: user@flume.apache.org

Looks like you have a protocol buffer version problem. Do you have a dependency on 2.5.0 version
of protocol buffers in your pom?

On Mon, Oct 28, 2013 at 4:02 PM, wesley dias <wesleydias@outlook.com> wrote:

I am using the latest flume build 1.4 along with hadoop 2.2.0 to collect twitter stream and
write to HDFS sink. However, on writing to HDFS I get the following error:
2013-10-28 09:41:23,481 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:422)]
process failed

java.lang.VerifyError: class org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$SetOwnerRequestProto
overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;

     at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
     at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)

    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

     at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
    at java.lang.Class.privateGetPublicMethods(Class.java:2651)

    at java.lang.Class.privateGetPublicMethods(Class.java:2661)
     at java.lang.Class.getMethods(Class.java:1467)
 I am currently new to flume and any help shall be really appreciated.



CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the individual or entity
to which it is addressed and may contain information that is confidential, privileged and
exempt from disclosure under applicable law. If the reader of this message is not the intended
recipient, you are hereby notified that any printing, copying, dissemination, distribution,
disclosure or forwarding of this communication is strictly prohibited. If you have received
this communication in error, please contact the sender immediately and delete it from your
system. Thank You. 		 	   		  
View raw message