Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 247C710428 for ; Tue, 29 Oct 2013 04:54:13 +0000 (UTC) Received: (qmail 50774 invoked by uid 500); 29 Oct 2013 04:54:12 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 50428 invoked by uid 500); 29 Oct 2013 04:54:07 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 50420 invoked by uid 99); 29 Oct 2013 04:54:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 04:54:05 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of wesleydias@outlook.com designates 65.54.190.80 as permitted sender) Received: from [65.54.190.80] (HELO bay0-omc2-s5.bay0.hotmail.com) (65.54.190.80) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 04:54:00 +0000 Received: from BAY178-W33 ([65.54.190.124]) by bay0-omc2-s5.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 28 Oct 2013 21:53:39 -0700 X-TMN: [NdrD8j9+rHe//D/yU8NhjfHR+uQJp9EO] X-Originating-Email: [wesleydias@outlook.com] Message-ID: Content-Type: multipart/alternative; boundary="_89049f41-bfd3-4d7b-a169-51da9be7c9fd_" From: wesley dias To: Roshan Naik , "user@flume.apache.org" Subject: RE: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink Date: Mon, 28 Oct 2013 23:53:38 -0500 Importance: Normal In-Reply-To: References: ,,, MIME-Version: 1.0 X-OriginalArrivalTime: 29 Oct 2013 04:53:39.0123 (UTC) FILETIME=[D5539030:01CED462] X-Virus-Checked: Checked by ClamAV on apache.org --_89049f41-bfd3-4d7b-a169-51da9be7c9fd_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Thanks Roshan. But issue is now resolved. I compiled flume 1.4 from source = using protobuf 2.5 and this solved the below mentioned issue. My guess is t= hat the distribution available for flume 1.4 from the release site must hav= e been compiled using protobuf 2.4 =20 Regards=2C =20 Wesley =20 Date: Mon=2C 28 Oct 2013 13:41:52 -0700 Subject: Re: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProce= ssor" on writing to HDFS sink From: roshan@hortonworks.com To: wesleydias@outlook.com HDP already includes hadoop 2.2.0 + Flume 1.4. You'd be better of just usin= g instead of building and testing that combo out on your own. http://hortonworks.com/products/hdp-2/?b=3D2#overview =0A= On Mon=2C Oct 28=2C 2013 at 1:19 PM=2C wesley dias = wrote: =0A= =0A= =0A= =0A= Hi David=2C =20 Thanks for the quick reply.Does this mean that I would need to compile the = flume from source using protobuf 2.5 instead of directly using the flume di= stribution available for download from the site? Is flume 1.4 based of prot= obuf 2.4? =0A= I am not sure of the same hence I am asking these questions. =20 Regards=2C =20 Wesley=20 Date: Mon=2C 28 Oct 2013 16:11:16 -0400 Subject: Re: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProce= ssor" on writing to HDFS sink =0A= From: dsinclair@chariotsolutions.com To: user@flume.apache.org =0A= Looks like you have a protocol buffer version problem. Do you have a depend= ency on 2.5.0 version of protocol buffers in your pom? see=20 http://code.google.com/p/protobuf/issues/detail?id=3D493 =0A= =0A= On Mon=2C Oct 28=2C 2013 at 4:02 PM=2C wesley dias = wrote: =0A= =0A= =0A= =0A= Hi=2C =20 I am using the latest flume build 1.4 along with hadoop 2.2.0 to collect tw= itter stream and write to HDFS sink. However=2C on writing to HDFS I get th= e following error: =20 2013-10-28 09:41:23=2C481 (SinkRunner-PollingRunner-DefaultSinkProcessor) [= ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java= :422)] process failed =0A= =0A= java.lang.VerifyError: class org.apache.hadoop.hdfs.protocol.proto.ClientNa= menodeProtocolProtos$SetOwnerRequestProto overrides final method getUnknown= Fields.()Lcom/google/protobuf/UnknownFieldSet=3B =0A= =0A= at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:1= 42) =0A= =0A= at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) =0A= =0A= at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) =0A= =0A= at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) =0A= =0A= at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2531) at java.lang.Class.privateGetPublicMethods(Class.java:2651) =0A= =0A= at java.lang.Class.privateGetPublicMethods(Class.java:2661) at java.lang.Class.getMethods(Class.java:1467) =20 I am currently new to flume and any help shall be really appreciated. =0A= =0A= =20 Regards=2C =20 Wesley =0A= =0A= =0A= =0A= =0A= CONFIDENTIALITY NOTICENOTICE: This message is intended for the use of the i= ndividual or entity to which it is addressed and may contain information th= at is confidential=2C privileged and exempt from disclosure under applicabl= e law. If the reader of this message is not the intended recipient=2C you a= re hereby notified that any printing=2C copying=2C dissemination=2C distrib= ution=2C disclosure or forwarding of this communication is strictly prohibi= ted. If you have received this communication in error=2C please contact the= sender immediately and delete it from your system. Thank You. = --_89049f41-bfd3-4d7b-a169-51da9be7c9fd_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Thanks =3BRoshan. But issue = is now resolved. I compiled flume 1.4 from source using protobuf 2.5 and th= is solved the below mentioned issue. =3BMy guess is that the distributi= on available for flume 1.4 from the release site must have been compiled us= ing protobuf 2.4
 =3B
Regards=2C
 =3B
Wesley
 = =3B
Date: Mon=2C 28 Oct 2013 13:41:52 -0700=
Subject: Re: Exception in thread "SinkRunner-PollingRunner-DefaultSinkP= rocessor" on writing to HDFS sink
From: roshan@hortonworks.com
To: we= sleydias@outlook.com

HDP already includes hadoop 2.= 2.0 + Flume 1.4. You'd be better of just using instead of building and test= ing that combo out on your own.


On Mon=2C Oct 28=2C 2013 at 1:19 PM=2C wesley dias <=3Bwesl= eydias@outlook.com>=3B wrote:
=0A=
=0A= =0A= =0A=
Hi David=2C
 =3B
Thanks for the quick reply= .Does this mean that I would need to compile the flume from source using pr= otobuf 2.5 instead of directly using the flume distribution available for d= ownload from the site? Is flume 1.4 based of protobuf 2.4?
=0A= I am not sure of the same hence I am asking these questions.
 =3BRegards=2C
 =3B
Wesley =3B
Date: Mon=2C 28 Oct 2= 013 16:11:16 -0400
Subject: Re: Exception in thread "SinkRunner-PollingR= unner-DefaultSinkProcessor" on writing to HDFS sink
=0A= From: d= sinclair@chariotsolutions.com
To: user@flume.apache.org

=0A=
Looks like you have a protocol buffer version problem. Do = you have a dependency on 2.5.0 version of protocol buffers in your pom?
see =3B




On Mon=2C Oct 28=2C 2013 at 4:= 02 PM=2C wesley dias <=3Bwesleydias@outlook.com>=3B wrote= :
=0A=
=0A= =0A= =0A=
Hi=2C
 =3B
I am using the latest flume buil= d 1.4 along with hadoop 2.2.0 to collect twitter stream and write to HDFS s= ink. However=2C on writing to HDFS I get the following error:
 =3B2013-10-28 09:41:23=2C481 (SinkRunner-PollingRunner-DefaultSinkPro= cessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:422)] process failed
=0A= =0A= java.lang.VerifyError: class org.apache.hadoop.hdfs.protocol.proto.<= u>ClientNamenodeProtocolProtos$SetOwnerRequestProto overrides fi= nal method getUnknownFields.()Lcom/google/protobuf/UnknownFie= ldSet=3B
=0A= =0A=  =3B =3B =3B at java.lang.ClassLoader.defineClass1(Nati= ve Method)
 =3B =3B =3B at java.lang.ClassLoader.defi= neClass(ClassLoader.java:800)
 =3B =3B =3B at java.se= curity.SecureClassLoader.defineClass(SecureClassLoader.java:1= 42)
=0A= =0A=  =3B =3B =3B at java.net.URLClassLoader.defineClass(URLC= lassLoader.java:449)
 =3B =3B =3B at java.net.URLCla= ssLoader.access$100(URLClassLoader.java:71)
 =3B = =3B =3B at java.net.URLClassLoader$1.run(URLClassLoader.java:361= )
=0A= =0A=  =3B =3B =3B at java.net.URLClassLoader$1.run(URLClassLo= ader.java:355)
 =3B =3B =3B at java.security.AccessCo= ntroller.doPrivileged(Native Method)
 =3B =3B =3B at= java.net.URLClassLoader.findClass(URLClassLoader.java:354)=0A= =0A=  =3B =3B =3B at java.lang.ClassLoader.loadClass(ClassLoa= der.java:425)
 =3B =3B =3B at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
 =3B =3B&nb= sp=3B at java.lang.ClassLoader.loadClass(ClassLoader.java:358= )
=0A= =0A=  =3B =3B =3B at java.lang.Class.getDeclaredMethods0(Nat= ive Method)
 =3B =3B =3B at java.lang.Class.privateGe= tDeclaredMethods(Class.java:2531)
 =3B =3B =3B at jav= a.lang.Class.privateGetPublicMethods(Class.java:2651)
=0A= =0A=  =3B =3B =3B at java.lang.Class.privateGetPublicMethods(= Class.java:2661)
 =3B =3B =3B at java.lang.Class.get= Methods(Class.java:1467)
 =3B
 =3BI am currently new t= o flume and any help shall be really appreciated.
=0A= =0A=  =3B
Regards=2C
 =3B
Wesley<= br>
=0A=

=0A=

=0A= =0A=
=0A= CONFIDENTIALITY NOTICE
NOTICE: This message is intended for t= he use of the individual or entity to which it is addressed and may contain= information that is confidential=2C privileged and exempt from disclosure = under applicable law. If the reader of this message is not the intended rec= ipient=2C you are hereby notified that any printing=2C copying=2C dissemina= tion=2C distribution=2C disclosure or forwarding of this communication is s= trictly prohibited. If you have received this communication in error=2C ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.
= --_89049f41-bfd3-4d7b-a169-51da9be7c9fd_--