hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10527) Fix incorrect return code and allow more retries on EINTR
Date Tue, 22 Apr 2014 22:08:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13977538#comment-13977538

Kihwal Lee commented on HADOOP-10527:

>  This doesn't seem like what you intended.
Rearranged the code. Since the terminal condition of the loop has changed (no more fixed limit
on # of iterations), I moved realloc() back to after the sys/lib calls.  Compared to pre-HADOOP-10522,
it uses single loop instead of nested loop and return code/error handling was modified.

> Also, please remove the comments about errno not being thread-safe.
They were removed in the previous patch. For a second, I though I screwed up and posted an
old patch again!

Here is something new about this patch.  

I thought about what the reasonable initial buffer size and the limit would be.  For user
lookups, the buffer can be relatively small and {{_SC_GETPW_R_SIZE_MAX}} won't be too big.
So the max buffer size here will mostly be for avoiding infinite lookup loop caused by a bug
in library.  I think 32KB is far more than enough for containing a single user's passwd entry.
So I set the buffer size limit to 32KB.  I did not change the initial buffer size as {{_SC_GETPW_R_SIZE_MAX}}
won't be excessively big and using it as the initial buffer size will avoid realloc() in most
cases, if not all.

Group lookups are different.The number of members in the group can vary a lot. {{_SC_GETGR_R_SIZE_MAX}}
is usually very big and allocating this much upfront may not be ideal.  So I set the initial
buffer to be 8KB and allowed it to grow up to 2MB. Assuming average user name length of 15
bytes, 8KB will be large enough to store a group with about 500 members. 2MB will be large
enough for a group with about 130K members.

What do you think about the new initial buffer size and the limit?

> Fix incorrect return code and allow more retries on EINTR
> ---------------------------------------------------------
>                 Key: HADOOP-10527
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10527
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>         Attachments: hadoop-10527.patch, hadoop-10527.patch, hadoop-10527.patch
> After HADOOP-10522, user/group look-up will only try up to 5 times on EINTR.  More retries
should be allowed just in case.
> Also, when a user/group lookup returns no entries, the wrapper methods are returning
EIO, instead of ENOENT.

This message was sent by Atlassian JIRA

View raw message