flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Shreedharan" <hshreedha...@cloudera.com>
Subject Re: FlumeTools/FileChannelIntegrityTool
Date Mon, 03 Nov 2014 19:03:06 GMT
You need to specify only the data directories(comma-separated), the channel will look at all
the files  in the directory.





Thanks,
Hari

On Mon, Nov 3, 2014 at 8:11 AM, Andy Yang <ayang_ml@yahoo.ca> wrote:

> Has anyone ever used the File Channel Integrity Tool released as part of Flume 1.4.0?
There is absolutely no documentation on it but anoutstanding JIRA issue from last year to
write some. 
>  
> I've been poking it with a stick as follows:
>  
> java -cp "/usr/lib/flume-ng/lib/*" org.apache.flume.tools.FlumeToolsMain FCINTEGRITYTOOL
-l /path/to/.flume/file-channel/data/log-xxx
>  
> but it returns far too quickly with no output, and no apparent change in the data file.
>  
> I'm hoping it will help with a corrupt file channel error:
>  
> java.lang.IllegalStateException: Channel closed [channel=c_xxx]. Due to java.io.IOException:
Unable to read next Transaction from log file /path/to/.flume/file-channel/data/log-xxx at
offset 513457598
> ...
> Caused by: com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol
message, the input ended unexpectedly in the middle of a field.  This could mean either than
the input has been truncated or that an embedded message misreported its own length.
> Thanks,
> Andy
Mime
View raw message