tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Masatake Iwasaki (Jira)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-4319) Add explicit dependency on snappy-java to tez-api
Date Wed, 14 Jul 2021 07:51:00 GMT

    [ https://issues.apache.org/jira/browse/TEZ-4319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17380410#comment-17380410
] 

Masatake Iwasaki commented on TEZ-4319:
---------------------------------------

We got error caused by unavalilable snappy library on BIGTOP-3573.
{noformat}
INFO impl.YarnClientImpl: Killed application application_1626227116175_0001,
org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found
for os.name=Linux and os.arch=aarch64,
    at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:460),       
    at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:318),     
    at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229),  
    at org.xerial.snappy.Snappy.<clinit>(Snappy.java:48),   
    at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79),    

    at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66),    
  
    at org.apache.tez.common.TezUtils.createByteStringFromConf(TezUtils.java:81),   
    at org.apache.tez.mapreduce.hadoop.MRInputHelpers.createMRInputPayload(MRInputHelpers.java:749),
       
    at org.apache.tez.mapreduce.input.MRInput$MRInputHelpersInternal.createMRInputPayload(MRInput.java:714),
 
    at org.apache.tez.mapreduce.input.MRInput$MRInputConfigBuilder.createGeneratorDataSource(MRInput.java:336),
    
    at org.apache.tez.mapreduce.input.MRInput$MRInputConfigBuilder.build(MRInput.java:266),
        
    at org.apache.tez.mapreduce.examples.TestOrderedWordCount.createDAG(TestOrderedWordCount.java:234),
      
    at org.apache.tez.mapreduce.examples.TestOrderedWordCount.run(TestOrderedWordCount.java:513),
  
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76),   
    at org.apache.tez.mapreduce.examples.TestOrderedWordCount.main(TestOrderedWordCount.java:624),
   
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method),         
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62),    
  
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
      
    at java.lang.reflect.Method.invoke(Method.java:498),      
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71),
      
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144),    
    at org.apache.tez.mapreduce.examples.ExampleDriver.main(ExampleDriver.java:86),   
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method),         
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62),    
  
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43),
      
    at java.lang.reflect.Method.invoke(Method.java:498),      
    at org.apache.hadoop.util.RunJar.run(RunJar.java:323),  
    at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
{noformat}


> Add explicit dependency on snappy-java to tez-api
> -------------------------------------------------
>
>                 Key: TEZ-4319
>                 URL: https://issues.apache.org/jira/browse/TEZ-4319
>             Project: Apache Tez
>          Issue Type: Improvement
>    Affects Versions: 0.10.1
>            Reporter: Masatake Iwasaki
>            Priority: Major
>
> Implicit dependency on snappy-java-1.0.5 was added by TEZ-4113. It causes error on platforms
like aarch64 for which snappy binary is not provided.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message