Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C99E1D19A for ; Tue, 11 Dec 2012 07:25:27 +0000 (UTC) Received: (qmail 91936 invoked by uid 500); 11 Dec 2012 07:25:26 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 91585 invoked by uid 500); 11 Dec 2012 07:25:26 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 90493 invoked by uid 99); 11 Dec 2012 07:25:24 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Dec 2012 07:25:24 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of nitinpawar432@gmail.com designates 209.85.217.179 as permitted sender) Received: from [209.85.217.179] (HELO mail-lb0-f179.google.com) (209.85.217.179) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Dec 2012 07:25:17 +0000 Received: by mail-lb0-f179.google.com with SMTP id gm13so2469908lbb.38 for ; Mon, 10 Dec 2012 23:24:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Rjc2LeeWwGdPv5bjSFqKQmdok9VfiZX+ZLnl222EHaM=; b=b2Q4NXrK+iDD5VjbwfxjykrE1CoI5YnQVPup7dsUDwEwigglYfmZK8ztLZBD3zW87g 47qE/Fnv5iSRkRhvqGmsMyMkrRGhHWmy5NnhaFbvQTS4hcvoTeKXK0stzU6Pm/YkwQO+ K6ByXR9dCICw81jh/lUEdY2ZSy1+uqx9CKNjKuwPnFijGlzyQkaQj537sCdRbuLEPp0/ TV2vz98BH0wJ1gtDMK2riTDKsVff/99222WeDQAUkHDRKnoqfKPu38BWjfSuaeFk5H3B Md42Zwie8OBFPkQspZ8BbpkpU0hCNIxSvE/2EiQ9XDeH89QP7YOJdXynEkLoRdn5rgFb dIfQ== MIME-Version: 1.0 Received: by 10.112.47.42 with SMTP id a10mr7225989lbn.30.1355210696637; Mon, 10 Dec 2012 23:24:56 -0800 (PST) Received: by 10.112.112.164 with HTTP; Mon, 10 Dec 2012 23:24:56 -0800 (PST) In-Reply-To: References: Date: Tue, 11 Dec 2012 12:54:56 +0530 Message-ID: Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1 From: Nitin Pawar To: shouvanik.haldar@accenture.com Cc: user@flume.apache.org Content-Type: multipart/alternative; boundary=bcaec554d7262b3fd204d08e9453 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec554d7262b3fd204d08e9453 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable there is no master/slave architecture now its a simple conf file like below <@EOM> and then just start the agent. it will do needful you need to start flume like the command below bin/flume-ng agent --conf-file example.conf --name agent1 agent1.sources =3D tail agent1.channels =3D MemoryChannel-2 agent1.sinks =3D HDFS agent1.sources.tail.type =3D exec agent1.sources.tail.command =3D tail -F agent1.sources.tail.channels =3D MemoryChannel-2 agent1.sources.tail.interceptors =3D hostint agent1.sources.tail.interceptors.hostint.type =3D org.apache.flume.interceptor.HostInterceptor$Builder agent1.sources.tail.interceptors.hostint.preserveExisting =3D true agent1.sources.tail.interceptors.hostint.useIP =3D true agent1.sinks.HDFS.channel =3D MemoryChannel-2 agent1.sinks.HDFS.type =3D hdfs agent1.sinks.HDFS.hdfs.path =3D hdfs://url:8020/user/flume/%{host} agent1.sinks.HDFS.hdfs.file.Type =3D SequenceFile agent1.sinks.HDFS.hdfs.writeFormat =3D Text agent1.sinks.HDFS.hdfs.rollInterval =3D 30 agent1.sinks.HDFS.hdfs.rollCount =3D 0 agent1.sinks.HDFS.hdfs.rollSize =3D 5000000000 agent1.channels.MemoryChannel-2.type =3D memory more examples at http://flume.apache.org/FlumeUserGuide.html On Tue, Dec 11, 2012 at 12:44 PM, wrote: > Hi,**** > > I installed flume 1.x higher version today! But is this very much > different from the earlier. ? Is there no master / node concept here? How > do I configure the source and sink?**** > > ** ** > > Think I got screwed here!**** > > ** ** > > *Thanks and Regards,* > > *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 * > > ** ** > > *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com] > *Sent:* Monday, December 10, 2012 3:34 PM > > *To:* Haldar, Shouvanik > *Cc:* user@flume.apache.org > *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO > com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1**** > > ** ** > > just a random question, does the user running flume agent have permission > to read the event details out of terracotta ? **** > > ** ** > > from the code here > http://grepcode.com/file/repository.cloudera.com/content/repositories/rel= eases/com.cloudera/flume-core/0.9.4-cdh4.1.0/com/cloudera/flume/watchdog/Wa= tchdog.java > **** > > ** ** > > line 209 onwards, it looks like the error condition is achieved when you > have channel set as stdin (I am not sure about this so you may need to lo= ok > at code which part of code you are hitting depending on the flume configs > you have) **** > > ** ** > > ** ** > > ** ** > > On Mon, Dec 10, 2012 at 3:01 PM, wrote:*= * > ** > > Hello,**** > > **** > > I am sure, the problem is due to Terracotta. Can you please help me? **** > > **** > > *Thanks and Regards,***** > > *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 ***** > > **** > > *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com] > *Sent:* Monday, December 10, 2012 2:43 PM > *To:* Haldar, Shouvanik > *Cc:* user@flume.apache.org**** > > > *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO > com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1**** > > **** > > I just hope you are not hitting this > https://issues.apache.org/jira/browse/FLUME-796**** > > I would also like to recommend you to upgraded to flume 1.x asap **** > > **** > > if possible can you try changing your flume source to some random file > with tail -f and try sinking it to some file. That will narrow the proble= m > is on flume side or terracotta side **** > > **** > > Thanks,**** > > Nitin **** > > **** > > On Mon, Dec 10, 2012 at 1:47 PM, wrote:*= * > ** > > Hi,**** > > **** > > I am using Flume -> Flume 0.9.4-cdh3u5. **** > > **** > > Initially, the logs are ok. But when I start terracotta > (./start-tc-server.sh) , then only I experience the above phenomena! Is i= t > something to do with terracotta.!**** > > **** > > **** > > 2012-12-10 03:14:08,806 INFO > com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking > thread pool server on port 35853...**** > > 2012-12-10 03:14:08,894 INFO net.sf.ehcache.terracotta.TerracottaClient: > Thread [logicalNode ec2-75-101-165-36.compute-1.amazonaws.com-19] > [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanceFactor= y > **** > > **** > > Before starting terracotta, Flume Node status is =93OPENING=94.**** > > But when I start terracotta, I got the error =93INFO > com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1=94**= ** > > **** > > Please help !**** > > **** > > **** > > *Thanks and Regards,***** > > *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 ***** > > **** > > *From:* Nitin Pawar [mailto:nitinpawar432@gmail.com] > *Sent:* Monday, December 10, 2012 12:52 PM > *To:* user@flume.apache.org > *Cc:* Haldar, Shouvanik > *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO > com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1**** > > **** > > what version of flume are you using? If it is < 1.x then recommend you to > upgrade to latest 1.0.3 **** > > If you are on 1.x then can you tell us whats the hostname for your server= ? > **** > > **** > > On Mon, Dec 10, 2012 at 11:40 AM, wrote:= * > *** > > Before the below mentioned error, I get a success message like**** > > **** > > 2012-12-10 01:09:14,403 INFO net.sf.ehcache.terracotta.TerracottaClient: > Thread [logicalNode ec2-75-101-165-36.compute-1.amazonaws.com-19] > [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanceFactor= y > **** > > **** > > *Thanks and Regards,***** > > *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 ***** > > **** > > *From:* Haldar, Shouvanik > *Sent:* Monday, December 10, 2012 11:34 AM > *To:* 'user@flume.apache.org' > *Subject:* Getting weird error -> 2012-12-10 01:01:42,190 INFO > com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1**** > > **** > > I had configured flume-master as well as flume-node on a single machine. = * > *** > > Now, as I run terracotta -> L1 cache, I get the INFO below in /**** > > **** > > 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: > Subprocess exited with value 1**** > > **** > > Can you please help? I am struggling with this for the past 1 day without > success!**** > > **** > > *Thanks and Regards,***** > > *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 ***** > > **** > > **** > ------------------------------ > > This message is for the designated recipient only and may contain > privileged, proprietary, or otherwise private information. If you have > received it in error, please notify the sender immediately and delete the > original. Any other use of the e-mail by you is prohibited. > > Where allowed by local law, electronic communications with Accenture and > its affiliates, including e-mail and instant messaging (including content= ), > may be scanned by our systems for the purposes of information security an= d > assessment of internal compliance with Accenture policy. > > > _________________________________________________________________________= _____________ > > www.accenture.com**** > > > > **** > > **** > > -- > Nitin Pawar**** > > > > **** > > **** > > -- > Nitin Pawar**** > > > > **** > > ** ** > > -- > Nitin Pawar**** > --=20 Nitin Pawar --bcaec554d7262b3fd204d08e9453 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable there is no master/slave architecture now=A0

its a simpl= e conf file like below <@EOM> and then just start the agent. it will = do needful=A0
you need to start flume like the command below=A0 

bin/flume-ng agent --conf-file example.conf --name agent1
<= br>

agent1.sources =3D tail
agent1.= channels =3D MemoryChannel-2
agent1.sinks =3D HDFS
agen= t1.sources.tail.type =3D exec
agent1.sources.tail.command =3D tail -F <file>
agent1.= sources.tail.channels =3D MemoryChannel-2
agent1.sources.tail.int= erceptors =3D hostint
agent1.sources.tail.interceptors.hostint.ty= pe =3D org.apache.flume.interceptor.HostInterceptor$Builder
agent1.sources.tail.interceptors.hostint.preserveExisting =3D true
agent1.sources.tail.interceptors.hostint.useIP =3D true
ag= ent1.sinks.HDFS.channel =3D MemoryChannel-2
agent1.sinks.HDFS.typ= e =3D hdfs
agent1.sinks.HDFS.hdfs.path =3D hdfs://url:8020/user/flume/%{host}
agent1.sinks.HDFS.hdfs.file.Type =3D SequenceFile
agent1.s= inks.HDFS.hdfs.writeFormat =3D Text
agent1.sinks.HDFS.hdfs.rollIn= terval =3D 30
agent1.sinks.HDFS.hdfs.rollCount =3D 0
agent1.sinks.HDFS.hdf= s.rollSize =3D 5000000000
agent1.channels.MemoryChannel-2.type = =3D memory




On Tue, Dec 1= 1, 2012 at 12:44 PM, <shouvanik.haldar@accenture.com>= wrote:

Hi,<= /p>

I installed flume 1.x hig= her version today! But is this very much different from the earlier. ? Is t= here no master / node concept here? How do I configure the source and sink?

=A0<= /p>

Think I got screwed here!=

=A0<= /p>

Thanks and Regards,

Shouvanik Haldar | Cloud = SME Pool | Mobile:+91-9830017568

=A0<= /p>

From: Nitin Pa= war [mailto:ni= tinpawar432@gmail.com]
Sent: Monday, December 10, 2012 3:34 PM


To: Haldar, Shouvanik
Cc: user@= flume.apache.org
Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO = com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1=

=A0

just a random question, does the user running flume = agent have permission to read the =A0event details out of terracotta ?=A0

=A0

=A0

line 209 onwards, it looks like the error condition = is achieved when you have channel set as stdin (I am not sure about this so= you may need to look at code which part of code you are hitting depending = on the flume configs you have)=A0

=A0

=A0

=A0

On Mon, Dec 10, 2012 at 3:01 PM, <shouvanik.haldar@accen= ture.com> wrote:

Hello,

=A0<= /p>

I am sure, the problem is= due to Terracotta. Can you please help me?

=A0<= /p>

Thanks and Regards,

Shouvanik Haldar | Cloud = SME Pool | Mobile:+91-98= 30017568

=A0<= /p>

From: Nitin Pa= war [mailto:ni= tinpawar432@gmail.com]
Sent: Monday, December 10, 2012 2:43 PM
To: Haldar, Shouvanik
Cc: user@= flume.apache.org


Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO = com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1=

=A0

I just hope you are not hitting this=A0https://is= sues.apache.org/jira/browse/FLUME-796

I would also like to recommend you to upgraded to fl= ume 1.x asap=A0

=A0

if possible can you try changing your flume source t= o some random file with tail -f and try sinking it to some file. That will = narrow the problem is on flume side or terracotta side=A0

=A0

Thanks,

Nitin=A0

=A0

On Mon, Dec 10, 2012 at 1:47 PM, <shouvanik.haldar@accen= ture.com> wrote:

Hi,<= /p>

=A0<= /p>

I am using Flume -> Fl= ume 0.9.4-cdh3u5.

=A0<= /p>

Initially, the logs are o= k. But when I start terracotta (./start-tc-server.sh) , then only I experie= nce the above phenomena! Is it something to do with terracotta.!=

=A0<= /p>

=A0<= /p>

2012-12-10 03:14:08,806 I= NFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread pool server on port 35853...

2012-12-10 03:14:08,894 I= NFO net.sf.ehcache.terracotta.TerracottaClient: Thread [logicalNode ec2-75-= 101-165-36.compute-1.amazonaws.com-19] [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanc= eFactory

=A0<= /p>

Before starting terracott= a, Flume Node status is =93OPENING=94.

But when I start terracot= ta, I got the error =93INFO com.cloudera.flume.watchdog.Watchdog: Subproces= s exited with value 1=94

=A0<= /p>

Please help !

=A0<= /p>

=A0<= /p>

Thanks and Regards,

Shouvanik Haldar | Cloud = SME Pool | Mobile:+91-98= 30017568

=A0<= /p>

From: Nitin Pa= war [mailto:ni= tinpawar432@gmail.com]
Sent: Monday, December 10, 2012 12:52 PM
To: user@= flume.apache.org
Cc: Haldar, Shouvanik
Subject: Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO = com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1=

=A0

what version of flume are you using? If it is < 1= .x then recommend you to upgrade to latest 1.0.3=A0

If you are on 1.x then can you tell us whats the hos= tname for your server?=A0

=A0

On Mon, Dec 10, 2012 at 11:40 AM, <shouvanik.haldar@acce= nture.com> wrote:

Before the below menti= oned error, I get a success message like

=A0

2012-12-10 01:09:14,40= 3 INFO net.sf.ehcache.terracotta.TerracottaClient: Thread [logicalNode ec2-= 75-101-165-36.compute-1.amazonaws.com-19] [cacheManager: 'RTLogCacheManager']: Creating new ClusteredInstanceFactory=

=A0

Thanks and Regards,

Shouvanik Haldar | Cloud = SME Pool | Mobile:+91-98= 30017568

=A0

From: Haldar, = Shouvanik
Sent: Monday, December 10, 2012 11:34 AM
To: '= user@flume.apache.org'
Subject: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.= cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1<= /u>

=A0

I had configured flume-master as well as flume-node = on a single machine.

Now, as I run terracotta -> L1 cache, I get the I= NFO below in /

=A0

2012-12-10 01:01:42,190 INFO com.cloudera.flume.watc= hdog.Watchdog: Subprocess exited with value 1

=A0

Can you please help? I am struggling with this for t= he past 1 day without success!

=A0

Thanks and Regards,

Shouvanik Haldar | Cloud = SME Pool | Mobile:+91-98= 30017568

=A0

=A0

This message is for the designa= ted recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the s= ender immediately and delete the original. Any other use of the e-mail by y= ou is prohibited.

Where allowed by local law, electronic communications with Accenture and it= s affiliates, including e-mail and instant messaging (including content), m= ay be scanned by our systems for the purposes of information security and a= ssessment of internal compliance with Accenture policy.

___________________________________________________________________________= ___________

www.accenture.com



=A0

--
Nitin Pawar



=A0

--
Nitin Pawar



=A0

--
Nitin Pawar




--
Nitin Pawar<= br>
--bcaec554d7262b3fd204d08e9453--