hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neil Ferguson (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10616) LoadSnappy should log the error message from UnsatisfiedLinkError
Date Sat, 17 May 2014 15:59:14 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Neil Ferguson updated HADOOP-10616:
-----------------------------------

    Description: 
org.apache.hadoop.io.compress.snappy.LoadSnappy currently does this:

{noformat}
    try {
      System.loadLibrary("snappy");
      LOG.warn("Snappy native library is available");
      AVAILABLE = true;
    } catch (UnsatisfiedLinkError ex) {
      //NOP
    }
{noformat}

So, the UnsatisfiedLinkError is swallowed without us ever knowing what the error message was.
However the message from UnsatisfiedLinkError can contain useful information for figuring
out why the library isn't loading. In my case it was:

"Exception in thread "main" java.lang.UnsatisfiedLinkError: snappy (/lib64/libc.so.6: version
`GLIBC_2.14' not found (required by /<path removed>/libsnappy.so))"

Telling me that snappy is built against the wrong version of libc.

I'd suggest logging the message from UnsatisfiedLinkError later on when we log whether snappy
was loaded or not. So, the full code for the static initializer be something like (not compiled
or tested):

{noformat}
  static {
    String message = null;
    try {
      System.loadLibrary("snappy");
      LOG.warn("Snappy native library is available");
      AVAILABLE = true;
    } catch (UnsatisfiedLinkError ex) {
      message = ex.getMessage();
    }
    boolean hadoopNativeAvailable = NativeCodeLoader.isNativeCodeLoaded();
    LOADED = AVAILABLE && hadoopNativeAvailable;
    if (LOADED) {
      LOG.info("Snappy native library loaded");
    } else {
      LOG.warn("Snappy native library not loaded");
      if (message != null) {
        LOG.warn("Unsatisified link error was [" + message + "]);
      }
    }
  }
{noformat} 

Though this could perhaps be a little neater.

  was:
org.apache.hadoop.io.compress.snappy.LoadSnappy currently does this:

{noformat}
    try {
      System.loadLibrary("snappy");
      LOG.warn("Snappy native library is available");
      AVAILABLE = true;
    } catch (UnsatisfiedLinkError ex) {
      //NOP
    }
{noformat}

So, the UnsatisfiedLinkError is swallowed without us ever knowing what the error message was.
However the message from UnsatisfiedLinkError can contain useful information for figuring
out why the library isn't loading. In my case it was:

"Exception in thread "main" java.lang.UnsatisfiedLinkError: snappy (/lib64/libc.so.6: version
`GLIBC_2.14' not found (required by /<path removed>/libsnappy.so))"

Telling me that snappy is built against the wrong version of libc.

I'd suggest logging the message from UnsatisfiedLinkError later on when we log whether snappy
was loaded or not. So, the full code for the static initializer be something like (not compiled
or tested):

{noformat}
  static {
    String message = null;
    try {
      System.loadLibrary("snappy");
      LOG.warn("Snappy native library is available");
      AVAILABLE = true;
    } catch (UnsatisfiedLinkError ex) {
      message = ex.getMessage();
    }
    boolean hadoopNativeAvailable = NativeCodeLoader.isNativeCodeLoaded();
    LOADED = AVAILABLE && hadoopNativeAvailable;
    if (LOADED) {
      LOG.info("Snappy native library loaded");
    } else {
      LOG.warn("Snappy native library not loaded");
      if (message != null) {
        LOG.warn("Unsatisified link error was [" + message + "]);
      }
    }
  }
{noformat} 

Or something like that.


> LoadSnappy should log the error message from UnsatisfiedLinkError
> -----------------------------------------------------------------
>
>                 Key: HADOOP-10616
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10616
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 0.23.10
>            Reporter: Neil Ferguson
>
> org.apache.hadoop.io.compress.snappy.LoadSnappy currently does this:
> {noformat}
>     try {
>       System.loadLibrary("snappy");
>       LOG.warn("Snappy native library is available");
>       AVAILABLE = true;
>     } catch (UnsatisfiedLinkError ex) {
>       //NOP
>     }
> {noformat}
> So, the UnsatisfiedLinkError is swallowed without us ever knowing what the error message
was. However the message from UnsatisfiedLinkError can contain useful information for figuring
out why the library isn't loading. In my case it was:
> "Exception in thread "main" java.lang.UnsatisfiedLinkError: snappy (/lib64/libc.so.6:
version `GLIBC_2.14' not found (required by /<path removed>/libsnappy.so))"
> Telling me that snappy is built against the wrong version of libc.
> I'd suggest logging the message from UnsatisfiedLinkError later on when we log whether
snappy was loaded or not. So, the full code for the static initializer be something like (not
compiled or tested):
> {noformat}
>   static {
>     String message = null;
>     try {
>       System.loadLibrary("snappy");
>       LOG.warn("Snappy native library is available");
>       AVAILABLE = true;
>     } catch (UnsatisfiedLinkError ex) {
>       message = ex.getMessage();
>     }
>     boolean hadoopNativeAvailable = NativeCodeLoader.isNativeCodeLoaded();
>     LOADED = AVAILABLE && hadoopNativeAvailable;
>     if (LOADED) {
>       LOG.info("Snappy native library loaded");
>     } else {
>       LOG.warn("Snappy native library not loaded");
>       if (message != null) {
>         LOG.warn("Unsatisified link error was [" + message + "]);
>       }
>     }
>   }
> {noformat} 
> Though this could perhaps be a little neater.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message