hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3579) libhdfs: fix exception handling
Date Mon, 06 Aug 2012 20:57:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429422#comment-13429422
] 

Aaron T. Myers commented on HDFS-3579:
--------------------------------------

bq. We need a longer running test that exercises more failure conditions to fully establish
that all memory leaks are fixed. I think writing such a test is a little bit of out scope
for this JIRA, but it's definitely something we should do in the future.

Definitely agree that writing such a test is well out of scope for this JIRA, but would it
be possible to, for example, run test_fuse_dfs with valgrind? (No need to do that for this
JIRA. This is good cleanup regardless, and we can fix any other memory leaks found in a different
JIRA.)

{quote}
Yes. Running a before and after with LIBHDFS_OPTS="-Xcheck:jni -Xcheck:nabounds" confirms
that the messages about "JNI call made with exception pending" are gone after the patch. The
test I ran was test_libhdfs_threaded.
I also ran test_fuse_dfs and verified that it passed successfully. That test also exercises
libhdfs, albeit in a slightly different way.
{quote}

Cool, thanks for doing that.

+1, I'll go ahead and commit this patch.
                
> libhdfs: fix exception handling
> -------------------------------
>
>                 Key: HDFS-3579
>                 URL: https://issues.apache.org/jira/browse/HDFS-3579
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: libhdfs
>    Affects Versions: 2.0.1-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-3579.004.patch, HDFS-3579.005.patch, HDFS-3579.006.patch
>
>
> libhdfs does not consistently handle exceptions.  Sometimes we don't free the memory
associated with them (memory leak).  Sometimes we invoke JNI functions that are not supposed
to be invoked when an exception is active.
> Running a libhdfs test program with -Xcheck:jni shows the latter problem clearly:
> {code}
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> Exception in thread "main" java.io.IOException: ...
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message