nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan Bende (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-6817) When there are multiple PutHDFS with snappy compression, will throw java.lang.UnsatisfiedLinkError
Date Mon, 28 Oct 2019 14:14:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961082#comment-16961082
] 

Bryan Bende commented on NIFI-6817:
-----------------------------------

The issue is a JVM limitation that limits a native library to being loaded by only one class
loader. The issue did become more noticeable when the HDFS processors moved to per-instance
class loading, instead of NAR class loading, but it would still be an issue across NARs. For
example, if the Hive NAR loads snappy and the HDFS NAR loads snappy, only one of them will
work.

> When there are multiple PutHDFS with snappy compression, will throw java.lang.UnsatisfiedLinkError
> --------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-6817
>                 URL: https://issues.apache.org/jira/browse/NIFI-6817
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.9.0, 1.9.2
>         Environment: nifi-1.9.2
> centos 7.3
>            Reporter: lixh
>            Priority: Major
>
> The first flow put hdfs use snappy compression codec work correct. now copy the flow
and start it, the second PutHDFS will throw  java.lang.UnsatisfiedLinkError.
> The hadoop native lib path config in bootstrap.conf: java.arg.17=-Djava.library.path=/usr/lib/hadoop/native
> I guess,Because AbstractHadoopProcessor.java has @RequiresInstanceClassLoading(cloneAncestorResources
= true),so org.apache.hadoop.util.NativeCodeLoader will be load by diffrent instanceClassloader



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message