hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3579) libhdfs: fix exception handling
Date Mon, 06 Aug 2012 21:03:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429427#comment-13429427
] 

Colin Patrick McCabe commented on HDFS-3579:
--------------------------------------------

Thanks, atm.  I have tried running valgrind on fuse_dfs in the past.  It doesn't really work--
I get tons of false positives.  I think there's a general problem running valgrind with JNI
code.  I did try adding more and more stuff to the "exclude lists," but it didn't seem to
work.  Maybe someone more knowledgeable on this topic can come up with a workaround.

I'm also confused about whether valgrind can identify memory leaks of memory managed by the
JVM.  I suspect that the answer is "no," which would mean that the local reference leaks fixed
by the patch would have been invisible to valgrind anyway.  As far as I know, valgrind only
deals with memory allocated via {{malloc}}-- although I'm happy to be corrected on this topic
if someone has more info ( ? )
                
> libhdfs: fix exception handling
> -------------------------------
>
>                 Key: HDFS-3579
>                 URL: https://issues.apache.org/jira/browse/HDFS-3579
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: libhdfs
>    Affects Versions: 2.0.1-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-3579.004.patch, HDFS-3579.005.patch, HDFS-3579.006.patch
>
>
> libhdfs does not consistently handle exceptions.  Sometimes we don't free the memory
associated with them (memory leak).  Sometimes we invoke JNI functions that are not supposed
to be invoked when an exception is active.
> Running a libhdfs test program with -Xcheck:jni shows the latter problem clearly:
> {code}
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> WARNING in native method: JNI call made with exception pending
> Exception in thread "main" java.io.IOException: ...
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message