flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Percy (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (FLUME-2427) java.lang.NoSuchMethodException and warning on HDFS (S3) sink
Date Fri, 17 Feb 2017 01:07:41 GMT

     [ https://issues.apache.org/jira/browse/FLUME-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mike Percy resolved FLUME-2427.
-------------------------------
       Resolution: Fixed
    Fix Version/s: v1.8.0

> java.lang.NoSuchMethodException and warning on HDFS (S3) sink 
> --------------------------------------------------------------
>
>                 Key: FLUME-2427
>                 URL: https://issues.apache.org/jira/browse/FLUME-2427
>             Project: Flume
>          Issue Type: Question
>            Reporter: Bijith Kumar
>            Assignee: Ping Wang
>            Priority: Minor
>             Fix For: v1.8.0
>
>         Attachments: FLUME-2427-0.patch
>
>
> The below warning and Exception is thrown every time a file is written to S3 using HDFS
sink. Looks like a jar mismatch to me. Tried latest hadoop and jets3 jars  but didn't work

> 17 Jul 2014 23:30:18,159 INFO  [hdfs-s3sink-engagements-call-runner-6] (org.apache.flume.sink.hdfs.AbstractHDFSWriter.reflectGetNumCurrentReplicas:184)
 - FileSystem's output stream doesn't support getNumCurrentReplicas; --HDFS-826 not available;
fsOut=org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream; err=java.lang.NoSuchMethodException:
org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsOutputStream.getNumCurrentReplicas()
> 17 Jul 2014 23:30:18,160 WARN  [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.hdfs.BucketWriter.getRefIsClosed:210)
 - isFileClosed is not available in the version of HDFS being used. Flume will not attempt
to close files if the close fails on the first attempt
> java.lang.NoSuchMethodException: org.apache.hadoop.fs.s3native.NativeS3FileSystem.isFileClosed(org.apache.hadoop.fs.Path)
> 	at java.lang.Class.getMethod(Class.java:1665)
> 	at org.apache.flume.sink.hdfs.BucketWriter.getRefIsClosed(BucketWriter.java:207)
> 	at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:295)
> 	at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:554)
> 	at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:426)
> 	at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> 	at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> 	at java.lang.Thread.run(Thread.java:744)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message