flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mahendran m <mahendra...@hotmail.com>
Subject RE: No threads used in Spooling Directory
Date Mon, 16 Feb 2015 11:27:59 GMT
This is what i wanted .waiting for flume 1.6.0 Release to use this features 😊
Thanks.

Date: Mon, 16 Feb 2015 16:26:25 +0530
Subject: Re: No threads used in Spooling Directory
From: paliwalashish@gmail.com
To: user@flume.apache.org

This could be of help https://issues.apache.org/jira/browse/FLUME-2413 (you need this at Sink
level but this should work as well)
HTH!


On Mon, Feb 16, 2015 at 4:01 PM, mahendran m <mahendranec@hotmail.com> wrote:



I am moving the IIS logs into remote hdfs using spooling directory and I need to update how
much data (size not no of events) has been moved successfully in real time in UI

Thanks,
Date: Mon, 16 Feb 2015 12:59:06 +0530
Subject: Re: No threads used in Spooling Directory
From: paliwalashish@gmail.com
To: user@flume.apache.org

What Flume Dev's meant was, Flume is not intended to do File transfers as a whole. A File
shall be read and an Event created out of the content based on how an Event needs to be created
(each line as an Event by default).
To answer you question (to the best of my knowledge)
Q1. I would like to know how much  file has been processed in parallel by SpoolDir ?
Nope
 
Q2. Is it possible to how much size of file has been moved by SpoolDir to sink  at any point
using built-in API?
Nope
If you can share what you want to achieve, we may be able to provide some pointers.
thanks
ashish



On Fri, Feb 13, 2015 at 5:43 PM, mahendran m <mahendranec@hotmail.com> wrote:



I need to move IIS logs to hdfs , then which is suitable for this purpose ?
Thanks ,

From: Manohar.CS@itcinfotech.com
To: user@flume.apache.org
Subject: RE: No threads used in Spooling Directory
Date: Fri, 13 Feb 2015 11:00:48 +0000









I had posted for similar requirement few days back , but from core team members got to know
that spoolDir is not suitable for fileBased handling, its more of
 event based content transfer that is being done.

 



From: mahendran m [mailto:mahendranec@hotmail.com]


Sent: Friday, February 13, 2015 11:02 AM

To: FlumeMaling List

Subject: No threads used in Spooling Directory



 


HI ,


 



Q1. I would like to know how much  file has been processed in parallel by SpoolDir ?



 



Q2. Is it possible to how much size of file has been moved by SpoolDir to sink  at any point
using built-in API?



 



Thanks , 







Please consider the environment before printing this e-mail



Disclaimer: This  communication  is  for the exclusive use of the intended recipient(s) and
 shall  not attach any liability on the originator or ITC Infotech India Ltd./its  Holding
company/ its Subsidiaries/ its Group Companies. If you are the addressee, the contents of
this e-mail are intended for your use only and it shall  not be forwarded to any third party,
without first obtaining written authorization from the originator or ITC Infotech India Ltd./
its Holding company/its  Subsidiaries/ its Group Companies. It may contain information which
is confidential and legally privileged and the same shall not be used or dealt with  by any
 third  party  in  any manner whatsoever without the specific consent  of  ITC  Infotech India
Ltd./ its Holding company/ its Subsidiaries/ its Group Companies. 		 	   		  


-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
 		 	   		  


-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
 		 	   		  
Mime
View raw message