hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LiXin Ge (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12094) Log torrent when none isa-l EC is used.
Date Thu, 06 Jul 2017 08:58:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

LiXin Ge updated HDFS-12094:
----------------------------
    Attachment: HDFS-12094.001.patch

> Log torrent when none isa-l EC is used.
> ---------------------------------------
>
>                 Key: HDFS-12094
>                 URL: https://issues.apache.org/jira/browse/HDFS-12094
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding
>    Affects Versions: 3.0.0-beta1
>            Reporter: LiXin Ge
>            Assignee: LiXin Ge
>            Priority: Minor
>         Attachments: HDFS-12094.001.patch
>
>
> My hadoop is built without isa-l support, after the EC policy is enabled, whenever I
get/put directory which contains many files, the log of warnings(see below) spam on the screen!
> This is unfriendly and depress the performance. Since we come to the beta version now,

> these logs should be deprecated and a one-time warning log instead of exception may be
much better.
> {quote}
> 2017-07-06 15:42:41,398 WARN erasurecode.CodecUtil: Failed to create raw erasure encoder
xor_native, fallback to next codec if possible
> java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawEncoder
> 	at org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawErasureCoderFactory.createEncoder(NativeXORRawErasureCoderFactory.java:35)
> 	at org.apache.hadoop.io.erasurecode.CodecUtil.createRawEncoderWithFallback(CodecUtil.java:177)
> 	at org.apache.hadoop.io.erasurecode.CodecUtil.createRawEncoder(CodecUtil.java:129)
> 	at org.apache.hadoop.hdfs.DFSStripedOutputStream.<init>(DFSStripedOutputStream.java:302)
> 	at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:309)
> 	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1216)
> 	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1195)
> 	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1133)
> 	...
> 	at org.apache.hadoop.fs.shell.Command.run(Command.java:176)
> 	at org.apache.hadoop.fs.FsShell.run(FsShell.java:326)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> 	at org.apache.hadoop.fs.FsShell.main(FsShell.java:389)
> 	
> Caused by: java.lang.RuntimeException: libhadoop was built without ISA-L support
> 	at org.apache.hadoop.io.erasurecode.ErasureCodeNative.checkNativeCodeLoaded(ErasureCodeNative.java:69)
> 	at org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawDecoder.<clinit>(NativeXORRawDecoder.java:33)
> 	... 25 more
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message