hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7017) Implement OutputStream for libhdfs3
Date Tue, 04 Nov 2014 18:47:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196585#comment-14196585

Colin Patrick McCabe commented on HDFS-7017:

Thanks, this looks better.

bq. I catch std::bad_alloc in lease renewer, if overcommit turned on, it does nothing, but
if it is thrown in some case, I do not want the library die in backend working thread. std::bad_alloc
will be thrown again somewhere in main thread and the API can handle it well.

I really can't agree with this rationale.  If {{std::bad_alloc}} is causing arbitrary threads
to terminate (without any message, since we don't log anything currently), how is the user
supposed to know?  And why do we think that "std::bad_alloc will be thrown again somewhere
in main thread"?  Perhaps terminating this thread freed up enough memory to proceed.

I think that 99.9999% of all users will run with memory overcommit turned on, which means
that this catch block will never be an issue.  The fact that nobody runs with overcommit disabled
also means this code will never be tested.

If we want to keep the catch block, let's at least log a message.  If you're concerned that
the logging will throw another exception, we can have another try... catch block.

> Implement OutputStream for libhdfs3
> -----------------------------------
>                 Key: HDFS-7017
>                 URL: https://issues.apache.org/jira/browse/HDFS-7017
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: Zhanwei Wang
>            Assignee: Zhanwei Wang
>         Attachments: HDFS-7017-pnative.002.patch, HDFS-7017-pnative.003.patch, HDFS-7017.patch
> Implement pipeline and OutputStream C++ interface

This message was sent by Atlassian JIRA

View raw message