hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LiXin Ge (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12094) Log torrent when none isa-l EC is used.
Date Thu, 06 Jul 2017 08:54:00 GMT
LiXin Ge created HDFS-12094:
-------------------------------

             Summary: Log torrent when none isa-l EC is used.
                 Key: HDFS-12094
                 URL: https://issues.apache.org/jira/browse/HDFS-12094
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: erasure-coding
    Affects Versions: 3.0.0-beta1
            Reporter: LiXin Ge
            Assignee: LiXin Ge
            Priority: Minor


My hadoop is built without isa-l support, after the EC policy is enabled, whenever I get/put
directory which contains many files, the log of warnings(see below) spam on the screen!
This is unfriendly and depress the performance. Since we come to the beta version now, 
these logs should be deprecated and a one-time warning log instead of exception may be much
better.
{quote}
2017-07-06 15:42:41,398 WARN erasurecode.CodecUtil: Failed to create raw erasure encoder xor_native,
fallback to next codec if possible
java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawEncoder
	at org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawErasureCoderFactory.createEncoder(NativeXORRawErasureCoderFactory.java:35)
	at org.apache.hadoop.io.erasurecode.CodecUtil.createRawEncoderWithFallback(CodecUtil.java:177)
	at org.apache.hadoop.io.erasurecode.CodecUtil.createRawEncoder(CodecUtil.java:129)
	at org.apache.hadoop.hdfs.DFSStripedOutputStream.<init>(DFSStripedOutputStream.java:302)
	at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:309)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1216)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1195)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1133)
	...
	at org.apache.hadoop.fs.shell.Command.run(Command.java:176)
	at org.apache.hadoop.fs.FsShell.run(FsShell.java:326)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
	at org.apache.hadoop.fs.FsShell.main(FsShell.java:389)
	
Caused by: java.lang.RuntimeException: libhadoop was built without ISA-L support
	at org.apache.hadoop.io.erasurecode.ErasureCodeNative.checkNativeCodeLoaded(ErasureCodeNative.java:69)
	at org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawDecoder.<clinit>(NativeXORRawDecoder.java:33)
	... 25 more
{quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message